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Abstract 

We  study  a  model  of  consumption  choice  and  portfolio  allocation  that  captures,  in  two  different 
interpretations,  the  combined  eifect  of  local  substitution  and  habit  formation  and  the  combined 
effect  of  durability  of  consumption  goods  and  habit  formation  over  service  flows  from  those 
goods.  In  a  third  interpretation,  the  model  captures  the  idea  of  a  dual  purpose  commodity. 
The  optimal  allocation  problem  is  from  the  class  of  free  boundary  singular  control  problems.  We 
discuss,  formally,  necessary  and  sufficient  conditions  for  a  consumption  and  portfolio  policy  to  be 
optimal.  We  aJso  introduce  a  numerical  technique  bjised  on  approximating  the  original  program 
by  a  sequence  of  discrete  parameter  Markov  chain  control  problems.  We  provide  convergence 
results  of  the  value  function,  the  optimal  investment  policy,  and  the  optimal  consumption 
regions  in  the  approximating  discrete  control  problems  to  those  in  the  original  continuous  time 
dynamic  program.  We  construct  numerically  the  consumption  boundary  that  divides  the  state 
space  into  two  regions —  one  of  immediate  consumption  and  the  other  of  abstinence.  We  show 
that  both  the  wealth  required  to  start  consuming  and  the  optimal  fraction  of  wealth  invested 
in  the  risky  asset  are  cyclical  functions  in  both  the  stock  of  the  durable  good  and  the  standard 
of  living.  This  is  due  to  the  interaction  between  the  durability  and  habit  formation  effects.  We 
also  study  the  effect  of  the  cyclical  investment  behavior  on  the  equilibrium  risk  premium  in  a 
representative  consumer  economy. 
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1     INTRODUCTION  AND  SUMMARY 


1      Introduction  and  Summary 


We  study  the  problem  of  optimal  consumption  and  portfolio  rules  for  an  agent  whose  preferences 
over  consumption  are  given  as 

(1)      E[J\-''ui2{t),y{t))dt], 
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where 


(2)  zit)     =     z{0-)e-'^^  +  f3  f    e-'^^'-'UC{s)     and 

(3)  y{t)     =     y(0-)e-^'  +  A  /'  e-^('-)dC(s). 

Jo- 


in this  formulation,  the  consumption  process  C{t)  denotes  the  total  amount  of  consumption  till 
time  t.  The  processes  z{t)  and  y{t)  are  derived  from  consumption  using  the  weighting  factors 
0  and  A,  respectively,  with  0  >  X.  Both  ^(0")  >  0  and  y{0~)  >  0  are  given  constants  and 
^  >  0  captures  the  impatience  of  the  agent.  The  felicity  function  u  is  continuous  and  strictly 
increasing.  Furthermore,  u  is  strictly  concave  and  has  the  property  that  Ui2  >  0,  where  Ui2  is 
the  second  cross  partial  derivative  of  u  with  respect  to  z  and  y. 

We  entertain  three  different  economic  ideas  in  three  different  interpretations  of  the  model 
specified  in  (1),  (2),  and  (3).  In  one  interpretation,  preferences  given  by  (1)  exhibit  the  notions 
of  local  substitution  and  habit  formation.  Agents  with  such  preferences  treat  consumptions 
at  adjacent  dates  as  close  substitutes  and  consumptions  at  distant  dates  as  complements.  In 
a  second  interpretation,  the  model  represents  habit  forming  preferences  over  the  service  flows 
from  irreversible  purchases  of  a  durable  good  that  decays  over  time.  In  the  third  interpretation, 
the  model  represents  preferences  for  consumption  of  a  dual  purpose  commodity  that  provides 
the  agent  with  two  sources  of  utility.  The  two  components  of  such  a  composite  good,  however, 
have  different  half-lives. 

Local  substitution  in  continuous  time,  analyzed  in  Hindy,  Huang,  and  Kreps  (1992),  Hindy 
and  Huang  (1992)  and  Heaton  (1993),  is  the  notion  that  consumption  at  one  time  reduces 
marginal  utility  at  nearby  times.  Habit  formation,  studied  since  Marshall  (1920),  is  the  notion 
that  agents  develop  tastes  because  of  past  consumption  experience.  Specifically,  a  high  standard 
of  living  in  the  past  increases  the  appetite  of  the  agent  for  current  consumption.  Habit  formation 
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has  been  studied  by  many  authors.  Ryder  and  Heal  (1973)  introduced  the  notion  of- adjacent 
versus  distant  complementariness.  Stigler  and  Becker  (1977)  emphasize  the  importance  ot 
analyzing  the  endogenous  development  of  preferences  in  the  search  for  factors  that  explain 
differences  in  tastes.  Abel  (1990),  Constantinides  (1990),  DeTemple  and  Zapatero  (1991), 
Heaton  (1993),  and  Sundaresan  (1989)  study  habit  formation  models. 

In  this  paper,  we  study  preferences  that  exhibit  a  combination  of  local  substitution  and 
habit  formation.  The  distinguishing  feature  of  the  preference  specification  in  (1)  is  that  the 
felicity  function  u  depends  only  on  exponentially  weighted  averages  of  past  consumption.  In 
particular,  the  current  consumption  rate  does  not  appear  directly  in  the  felicity  function.  This 
feature,  which  is  absent  in  almost  all  non-time-additive  preferences  in  the  literature,  is  the  key 
to  representing  the  notion  of  local  substitution.  For  the  details,  we  refer  the  reader  to  Hindy, 
Huang,  and  Kreps  (1992)  and  Hindy  and  Huang  (1992). 

Preferences  of  representative  consumers  have  been  used,  in  general  equilibrium  models,  to 
explain  the  behavior  of  the  returns  on  financial  assets.  Recent  empirical  studies  of  Eichenbaum 
and  Hansen  (1990),  Gallant  and  Tauchen  (1989),  and  Heaton  (1993)  suggest  that  there  is 
substitution  over  short  periods  and  habit  formation  over  long  periods.  In  particular,  Heaton 
(1993)  showed,  after  correcting  for  the  problem  of  temporal  aggregation,  that  habit  formation 
alone  does  not  provide  significant  explanation  power  for  asset  pricing  over  the  time-additive 
models;  while  local  substitution,  or  durability,  does.  In  addition,  given  local  substitution,  or 
durability,  habit  formation  over  long  horizons  becomes  more  significant  in  its  explanatory  power. 
These  results  imply  that  a  model  which  combines  local  substitution  and  habit  formation  may 
better  explain  the  behavior  of  security  returns.  The  analysis  of  the  optimization  problem  for  a 
single  consumer  is  an  important  step  in  that  direction. 

In  the  durable  good  version  of  the  model,  we  identify  C{t)  with  the  total  purchases  of 
a  durable  good  till  time  t.  Purchases  of  the  good  are  irreversible  either  because  there  is  no 
secondary  market  for  the  good,  or  because  of  prohibitively  high  selling  costs.  We  invite  the 
reader  to  think  of  such  durables  as  grocery,  cloth,  small  appliances,  and,  even,  vacations.  The 
good  provides  a  flow  of  services  proportional  to  the  stock  z{t)  of  the  good  at  time  t.  Absent  any 
new  purchases,  the  service  flow  declines  over  time  because  of  the  deterioration  in  the  stock  of  the 
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durable  good.  The  standard  of  living  of  the  agent  is  given  by  y{t)  and  reflects  past  consumption 
experience.  We  take  it  that  the  standard  of  living  deteriorates  at  a  much  slower  rate  than  the. 
stock  of  the  durable.  This  assumption  is  reflected-in  the  restriction  that  /?  >  A.  Discrete  time 
versions  of  this  model  have  been  studied  by  Dunn  and  Singleton  (1986),  Eichenbaum,  Hansen 
and  Singleton  (1988)  and  Hotz,  Kydland  and  Sedlacek  (1988),  among  others. 

In  the  third  interpretation  of  the  model,  we  take  C{t)  to  be  the  total  purchases,  till  time 
f,  of  a  composite  commodity.  This  commodity  provides  the  agent  with  two  sources  of  utility. 
For  example,  consider  food  that  provides  "calories"  and  "vitamins",  or  cloth  that  provides 
"shelter"  and  "style".  The  two  components  of  the  dual  purpose  commodity  are  captured  by 
z{t)  and  y{t).  The  half-lives  of  the  two  components  are  different.  Furthermore,  the  two  sources 
of  satisfaction  are  complementary.  For  example,  a  higher  level  of  energy  increases  the  marginal 
utility  of  vitamins.  This  feature  is  captured  by  the  assumption  that  ui2  >  0.  In  the  balance 
of  the  paper  we  will  use  the  terminology  "consumption"  and  "purchases  of  the  durable  good" 
interchangeably. 

The  specification  of  preferences  in  (1)  leads  to  very  interesting  behavior  as  well  as  a  tech- 
nically challenging  optimization  problem.  From  the  analysis  in  Hindy  and  Huang  (1993),  who 
study  a  model  without  habit  formation,  it  is  known  that  the  optimization  problem  specified 
here  belongs  to  the  class  of  free  boundary  singular  control  problems.  The  main  feature  of  the 
problem  is  that  consumption  occurs  periodically.  The  agent  consumes  only  when  the  marginal 
utility  of  wealth  is  equal  to  a  linear  combination  of  the  marginal  utility  of  the  stock  of  the 
durable  good  and  the  marginal  utility  of  the  standard  of  living.  This  condition  is  satisfied  only 
for  a  particular  combination  of  wealth,  stock  of  the  durable,  and  standard  of  living.  Search- 
ing for  this  combination,  or  free  boundary,  is  the  essence  of  solving  the  utility  maximization 
problem. 

We  first  discuss,  formciDy,  necessary  and  sufficient  conditions  for  a  consumption-investment 
policy  to  be  optimal.  In  particular,  we  provide  a  verification  theorem  that  shows  that  the 
value  function  —  the  maximum  attainable  utility  starting  from  any  initial  position  —  satisfies 
a  differential  inequality.  Such  differential  inequalities  are  the  hallmark  of  free  boundary  con- 
trol problems.   The  verification  theorems  we  provide  require  that  the  value  function  be  twice 
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continuously  differentiable  in  W  and  once  continuously  differentiable  in  z  and  y.  -  .      u,  .„ 

In  general,  it  is  difRciilt  to  ascertain  the  smoothness  of  the  value  function.  In  Hindy  and, 
Huang  (1993)  such  smoothness  could  be  established  because  there  is  a  closed-form  expression 
for  the  value  function.  In  the  problem  we  study  in  this  paper,  such  a  closed-form  solution  is 
not  currently  available.  For  this  reason,  our  numerical  analysis  relies  on  a  weaker  notion  for 
the  solution  of  the  dynamic  programming  equation.  Such  notion,  known  as  viscosity  solution, 
is  described  in  details  in  a  companion  paper  by  Hindy,  Huang,  and  Zhu  (1993).  We  record  the 
verification  theorems  for  smooth  value  functions  for  two  reasons.  First,  the  discussion  is  useful 
for  conveying  the  economic  intuition  of  the  optimal  soliltion.  Second,  the  verification  theorems 
can  be  utilized  in  the  future  if  a  closed-form  solution  is  obtained. 

We  describe  a  numerical  approach  for  solving  the  utility  maximization  problem.  The  idea 
of  the  numerical  approach  is  to  approximate  the  controlled  processes,  which  are  continuous- 
time  diffusions,  by  appropriately  chosen  Markov  chains  on  a  finite  state  space.  In  addition,  we 
approximate  the  utility  function  in  (1)  by  one  which  is  appropriate  for  the  Markov  chain.  The 
approximating  Markov  chains  are  chosen  to  satisfy  a  "local  consistency"  condition.  This  condi- 
tion is,  roughly,  that  the  conditional  expected  change  in  the  Markov  chain  and  its  conditional 
variance  locally  match  the  drift  and  variance  of  the  original  controlled  process. 

We  approximate  the  decision  problem  we  study  here  by  a  sequence  of  Markov  chain  control 
problems.  Each  Markov  chain  control  problem  in  the  sequence  is  readily  solvable  numericallyi 
We  then  show  that  the  value  functions  in  the  sequence  of  approximating  control  problems 
converge  to  the  value  function  of  the  original  controlled  diffusion  problem.  We  remark  that 
existence  of  the  value  function  and  the  feasibility  of  computing  its  values  with  high  accuracy 
do  not  guarantee  existence  of  optimal  policies  that  achieve  the  value  function.  Contrast  this 
with  classical  feedback  control  case.  In  that  case,  optimal  controls  can  be  expressed  as  explicit 
functions  of  the  value  function  and  its  derivatives.  As  a  consequence,  existence  of  a  smooth 
value  function  immediately  implies  existence  of  optimal  controls.  In  our  case,  with  a  free- 
boundary  problem,  there  is  no  direct  relationship  between  the  value  function  and  the  optimal 
controls,  which,  if  exist,  are  not  feedback. 
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We,  hence,  show  that  the  sequence  of  optimal  investment  policies  and  optimal  consumption 
regions  in  the  approximating  Markov  chain  problems  converge  to  the  optimal  solutions  in  the 
originaJ  problem,  if  the  latter  exist.  In  this  paper,  we  only  sketch  without  proof  the  numerical 
technique  and  the  convergence  results.  The  companion  paper,  Hindy,  Huang,  and  Zhu  (1993) 
provides  the  details  and  proofs. 

We  provide  numerical  solutions  for  the  infinite  horizon  maximization  problem  when  the 
price  of  the  risky  assets  follow  geometric  Brownian  Motion.  We  analyze  a  felicity  function 
u{z,y,t)  of  the  multiplicative  form  e'^'z^'j/"^  for  some  constants  S,ai,  and  03.  Given  the 
convergence  results,  all  the  numerical  solutions  we  discuss  here  can  be  made  arbitrarily  close  to 
the  optimal  solution  of  the  original  problem.  In  particulaj,  the  regions  of  optimal  consumption 
and  the  optimal  investment  policies  we  report  are  very  good  approximations  of  the  optimal 
policies  in  the  continuous  time  problem,  if  the  latter  exist.  For  ease  of  exposition,  we  will  call 
the  solutions  we  report  here  "optimal".  The  reader  should  remember  that  such  solutions  are 
rather  e-optimal  in  the  sense  that  they  are  very  close  approximations  to  the  optimal  solution. 

We  compute  the  optimal  consumption  boundary  W*{z,y).  An  agent  with  wealth  W,  stock 
of  the  durable  good  z  and  standard  of  living  y  such  that  W  <  W'(z,y)  optimally  refrains  from 
consumption.  Such  an  agent  waits  as  wealth  increases,  on  average,  and  both  z  and  y  decline 
till  the  first  time  that  the  trajectory  of  the  "state  variables"  {W,  2,  y)  hits  the  consumption 
boundary.  An  agent  with  state  variables  such  that  W  >  W'{z,y)  optimally  consumes  an 
amount  of  consumption  that  reduces  wealth  and  increases  z  and  y  instantly  to  bring  the  state 
variables  to  the  consumption  boundary.  Once  on  the  consumption  boundary,  an  agent  consumes 
the  minimum  amount  required  to  keep  the  state  variables  from  crossing  the  boundary. 

The  striking  feature  of  the  solution  is  that  the  consumption  boundary  W'{z,y)  is  cyclical 
as  a  function  of  z  and  y.  For  a  fixed  standard  of  living  y,  the  critical  wealth  required  for 
an  agent  to  start  consuming  is  not  a  monotone  function  in  the  stock  of  the  durable  good  2. 
Instead,  it  increases  for  a  while  with  increases  in  z,  then  declines  with  further  increases  in  z, 
and  then  reverses  its  trend  and  increases  again  with  increases  in  z.  Similarly,  W'{z,y)  is  a 
cyclical  function  of  the  standard  of  living  y  for  fixed  z. 
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The  optimal  investment  policy  A*(iy,  z,  y) —  the  proportion  of  wealth  invested  in  the  risky 
assets —  follows  a  similar  cyclical  pattern  as  both  the  stock  of  the  durable  good  and  the  standard 
of  living  change.  In  particular,  the  partial  derivatives  ^^  and  ^^  change  their  sign  periodically 
as  z  and  y,  respectively,  change.  The  cyclical  behavior  of  the  critical  wealth  level  and  the 
investment  policy  contrasts  with  the  behavior  of  an  agent  in  the  presence  of  local  substitution, 
or  durability,  but  without  habit  formation.  In  this  case,  studied  in  Hindy  and  Huang  (1993), 
the  critical  wealth  required  to  begin  consumption  is  linear  in  the  stock  of  the  durable  good. 
Moreover,  the  proportion  optimally  invested  in  the  risky  asset  is  constant. 

The  cyclical  pattern  in  the  consumption  boundary  and  the  investment  policy  results  from 
the  interaction  of  the  durability  and  habit  formation  effects.  An  additional  unit  of  the  durable 
good  has  two  conflicting  effects.  The  first  is  a  direct  satiating  effect.  The  agent's  appetite  for 
more  of  the  durable  good  is  reduced.  The  second  is  an  indirect  stimulating  effect.  As  the  stock 
of  the  durable  good  increases,  the  agent's  appetite  for  a  higher  standard  of  living  increases. 
This  is  the  effect  of  complementarity  or  habit  formation.  As  a  result,  the  agent  would  like  to 
consume  more  of  the  good  to  increase  the  standard  of  living.  The  satiating  and  stimulating 
effects  also  influence  the  investment  behavior  of  the  agent.  When  the  satiating  effect  dominates, 
the  agent  invests  a  high  fraction  of  wealth  in  the  risky  assets  since  he  can  afford  to  tolerate 
high  losses  in  wealth.  On  the  other  hand,  when  the  stimulating  effect  is  dominant,  the  agent 
invests  a  smaller  fraction  of  wealth  in  the  risky  asset.  The  agent  in  this  case  behaves,  in  a  more 
risk  averse  manner  to  protect  the  standard  of  living. 

The  interesting  feature  of  the  solution  is  that  the  relative  strength  of  the  satiating  and 
stimulating  effects  alternates  as  z  ajid  y  change.  We  document  the  conflict  between  the  two 
effects  of  consumption  in  a  series  of  graphs  that  display  the  variations  of  marginal  utilities  of  the 
stock  of  the  durable  and  the  standard  of  living  as  the  state  variables  vary.  We  also  discuss  the 
long  term  behavior  of  optimally  invested  wealth  and  optimally  followed  standard  of  living  and 
stocks  of  durable  goods  in  a  population  of  agents.  As  a  consequence  of  the  cyclical  nature  of 
the  consumption  boundary,  initial  differences  in  life  style —  the  ratios  z/W  and  y/W —  between 
agents  who  are  identical  in  preferences  persist  indefinitely.  In  essence,  agents  in  the  population 
are  divided  into  distinct  life-style  classes.  Members  of  the  same  class  eventually  adopt  the  same 
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life  style.  However,  a  member  of  one  class,  behaving  optimally,  does  not  migrate  to  another 
class.  This  contrasts  with  the  anailysis  of  Constantinides  (1990)  and  Hindy  and  Huang  (1993) 
in  which  the  ratio  of  the  stock  of  the  durable  to  wealth  converges  to  a  steady  state  distribution 
regardless  of  the  initial  endowments. 

We  also  study  the  implications  of  our  model  on  the  determination  of  the  risk  premium. 
We  compute  the  risk  premium  in  an  economy  with  constant  stochastic  returns  to  scale.  We 
show  that  the  interaction  between  the  satiating  and  stimulating  effects  of  consumption  leads  to 
cyclical  movement  in  the  risk  premium.  In  the  stationary  investment  environment  we  study,  the 
risk  premium  changes  cyclically  because  the  attitudes  of  the  representative  consumer  towards 
risk  change  in  cycles.  The  culprit,  of  course,  is  the  interaction  between  durability,  or  local 
substitution,  with  habit  formation. 

Finally,  we  remark  that  our  analysis  is  confined  to  the  decision  of  an  individual  to  purchase 
a  durable  good.  Constructing  aggregate  time  series  of  purchases  in  an  economy  populated  with 
many  agents  of  the  type  we  study  is  important  for  understanding  the  relationships  between 
aggregate  consumption  and  asset  returns.  Such  an  interesting  study,  however,  is  beyond  the 
scope  of  this  paper. 

The  rest  of  the  paper  is  organized  as  follows.  Section  2  describes  the  setup  of  the  optimiza- 
tion problem.  The  necessary  and  sufficient  conditions  for  optimality  are  discussed  in  section  3. 
The  numerical  technique  and  its  implementation  are  discussed  in  sections  4  and  5,  respectively. 
Section  6  reports  an  example  of  the  numerical  solution  of  an  infinite  horizon  program  when  the 
price  of  the  risky  asset  follows  a  geometric  Brownian  motion.  The  implications  of  the  solution 
on  the  determination  of  the  risk  premium  on  risky  assets  in  equilibrium  is  discussed  in  section  7. 
Section  8  contains  some  concluding  remarks. 

2      Formulation 

Consider  an  agent  who  lives  from  time  <  =  Otof  =  ooina  world  of  uncertainty  where  there  is 
a  single  good  available  for  consumption  at  any  time.  The  agent  has  the  opportunity  to  invest 
in  a  frictionless  securities  market  with  N  +  I  long  lived  securities,  continuously  traded,  and 
indexed  by  n  =  0, 1,2,...,  A'^.  Security  n,  where  n  =  1,2, . . .,  jV,  is  risky,  pays  dividends  at  rate 
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Pn{t),  and  sells  for  5„(<)  at  time  t.  We  assume  that  />„(<)  can  be  written  as  pnC'S'CO))  where 
we  have  used  S{t)  to  denote  the  column  vector  [5i(t),  S2{t), . . .,  5;v(0]  ^ ■  Security  0  is  locally 
riskless,  does  not  pay  dividend,  and  sells  for  B{t)  =  exp{/Q  r(5(5),s)d5}  at  time  t,  where  r{S) 
is  the  instantaneous  riskless  interest  rate  at  time  t  and  r(-)  is  Borel  measurable. 

The  price  process  for  the  risky  securities  follows  a  diffusion  process  given  by:  ■  .     .. 


(4)      5(0  +  /'  p{S{s))  ds  =  5(0)  +  /'  fiiS{s))  ds  +  /'  a{S{s))  dBis)    Vt  €  [0,  oo) 
Jo  Jo  Jo 


a.s.. 


where  B  is  an  M-dimensionaJ,  M  >  N ,  standard  Brownian  motion  defined  on  a  complete  prob- 
ability space  {Cl,^,  P).^  Assume  that  this  diffusion  process  is  strictly  positive  with  probability 
one  and  that  for  each  integer  m  >  0,  there  exists  a  constant  c^  so  that  f^[|5(<)|^'"]  <  e*^"*'.^ 

We  assume  that  the  agent  has  only  access  to  the  information  contained  in  the  historical 
prices  of  the  risky  securities.  We  denote  this  information  structure  by  F  =  {^ui  S  [0,oo)}, 
where  Tt  is  the  smallest  sub-sigma-field  of  ^  with  respect  to  which  {5(a);  0  <  a  <  <}  is 
measurable.  We  assume  that  .Fj  contains  aU  the  probability  zero  sets  of  /",  or  F  is  complete. 
All  processes  to  be  discussed  will  be  adapted  to  F.* 

The  agent  can  consume  the  single  good  at  "gulps"  at  any  moment,  and  can  consume  at 
finite  rates  over  intervals.  The  agent  can  also  refrain  from  consumption  altogether  for  some 
time.  Moreover,  the  sample  path  of  cumulative  consumption  at  any  time  t  can  have  a  singular 
component,  that  is  a  continuous  nontrivial  increasing  function  whos6^  derivative  is  zero  for 
almost  all  t. 

Let  X+-be  the  space  of  all  processes  x  whose  sample  paths  are  positive^,  increasing  and 
right-continuous.   Recall  that  an  increasing  function  x{u), .)  has  a  finite  left -limit  at  any  t  G 


'The  superscript  ^  denotes  transpose. 

^A  process  V  is  a  mapping  Y.ii  x  [0,  oo)  — •  ?J  that  is  measurable  with  respect  to  Z"®  B([0,  oo)),  the  product 
sigma-field  generated  by  !F  and  the  Borel  sigma-field  of  [0,  oo).  For  each  u  £  Q,  Y{iji, .):  [0,  oo)  — ►  R  is  a  sample 
path  and  for  each  t  £  [0, oo),  Y{.,t):U  — ►  St  is  a  random  variable.  We  will  use  the  following  notation:  If  ;x  is  a 
vector  in  St",  let  |/i|  be  the  Euclidean  norm  of  /x.  In  addition,  if  <t  is  a  matrix,  let  \<j\^  denote  tr((T<r  ),  where 
tr  is  the  trace  of  a  square  matrix.  The  notation  a.s.  denotes  statements  which  are  true  with  probability  one. 
For  brevity,  we  will  sometimes  use  /i(t),  p{t),  ff(<),  and  r(t)  to  denote  /i(5(t)),  p{S{t)),  <t(5(<)),  and  r{S{t)), 
respectively. 

'The  latter  can  be  ensured  by  a  growth  condition  on  [ft  —  p)  and  on  <r;  see  Friedman  (1975,  theorem  5.2.3). 

*The  process  Y  is  said  to  be  adapted  to  F  if  for  each  t  £  [0,  oo),  Y{t)  is  J'l-measurable.  This  is  a  natural 
information  constraint:  the  value  of  the  process  at  time  t  cannot  depend  on  information  yet  to  be  revealed. 

^We  use  weak  relations.   Positive  means  nonnegative  and  increasing  means  nondecreasing. 
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(0,00)  denoted  by  i(a;,t~).  We  will  use  the  convention  that  x{u,0~)  =  0  a. 5.  Since  left  limits 
exist  for  the  sample  paths  of  any  x  €  X+,  a  jump  of  i(u;, .)  at  r  is  Ai(w,r)  =  x{lj,t)-x{u>,t~). 

The  stochastic  process  C  £  X+  is  a  consumption  pattern  available  to  the  agent  with  C{u,t) 
denoting  the  cumulative  consumption  from  time  0  to  time  t  in  state  w.  For  any  a;  G  fi,  the 
points  of  discontinuity  of  C(a',  t)  are  the  moments  when  the  agent  consumes  a  "gulp".  Moreover, 
C{u),  t)  has  an  absolutely  continuous  component  over  the  intervals  during  which  the  agent  is 
consuming  at  rates  C'{u,t),  where  C'{u;,t)  denotes  the  consumption  rate  at  time  t  in  state  lj. 
Finally,  C{u},t)  may  have  a  singular  part. 

An  investment  strategy  is  an  A'^-dimensional  process  A  =  {A{t)  =  {Ai{t),. . .,  A!^{t));t  £ 
[0,  00)},  where  An{t)  denotes  the  proportion  of  wealth  invested  in  the  n-th  risky  security  at  time 
t  before  consumption  and  trading.  The  proportion  invested  in  the  riskless  security  is  1  — A(i)^l, 
where  1  is  a  vector  of  I's.  A  consumption  plan  C  G  X^  is  said  to  be  financed  by  an  investment 
strategy  A  if 

(5^        mt)     =  mO)  +  lo{ms)r{s)  +  W{s)A^{3)Is-r{s){ti{s)  -  r(.)5(.)))  ds  -  C{t-) 

+  /(J  W{s)A'^{s)Is-i  {s)(t{s)  dB{s),         Wt  €  [0,  00)     a.s. 
where  W{t)  is  wealth  at  time  t  before  consumption  and  Is-\{t)  is  an  N  X  N  diagonal  matrix 
with  the  n-th  element  on  the  diagonal  equal  to  S{t)~^.    Note  that  the  wealth  process  has 
left-continuous  sample  paths  and  that  W{t'^)  =  W{t)  —  AC{t). 

The  agent  derives  satisfaction  from  past  consumption  and  from  final  wealth  and  utility  is 
given  by  (1).  The  felicity  function  at  time  t  is  defined  over  the  stock  of  durable,  z,  and  the 
standard  of  living,  y,  defined,  respectively,  as 

(6)  z(0     =     ziO-)e-'^^  +  (3  f*  e-^^'-'UCis)    a.s., 

Jo- 

(7)  y(0     =     y(0-)e-^'  +  A  /'  e-^('-')dC(3)     a.s., 

No- 
where 2(0")  >  0  and  ^(O")  >  0  are  constants,  /?  and  A,  with  /?  >  A,  are  weighting  factors,  and 
the  integrals  in  (6)  and  (7)  are  defined  path  by  path  in  the  Lebesgue-Stieltjes  sense.  Note  that 
the  lower  limit  of  the  integrals  in  (6)  and  (7)  is  0~,  to  account  for  the  possible  jump  of  C  at 
t  =  0.  Also  note  that  z  and  y  are  right  continuous  processes  which  jump  whenever  C  does  and 
have  singular  component  whenever  C  does. 
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A  consumption  plan  C  and  the  investment  strategy  A  that  finances  it  are  said  to  be  a,dmis- 
sible  if  (5)  is  well-defined^,  E[J^  e~^*\u{z{t),y{t))\dt]  <  oo,  where  z  and  y  are  associated  with 
C,  and  for  all  integers  m  >  0  and  T  €  3*?+,  there  ejcists  k^  so  that,  for  all  t  <  T,        '  .      -       . 

(8)  E[\Cit)\'^"']  <  e*=-'     and     E[|M^(01""]  <  e*-'. 

Denote  by  C  and  A  the  space  of  admissible  consumption  plans  and  trading  strategies,  respec- 
tively. Formally,  the  agent  manages  wealth  dynamically  to  solve  the  following  program: 

supcec     E[j^e-''nizit),yit))dt\ 

(9)  s.t.      C  is  financed  hy  A  £  A  with  W{0)  =  Wq, 

and  W{t)  -  AC{t)  >  0    Vt  €  [0,  oo), 

where  Wq  is  the  initial  wealth  of  the  agent  and  u  is  the  felicity  function  at  time  t.  Note  that 
the  second  constraint  of  (9)  is  a  positive  wealth  constraint  —  wealth  after  consumption  at  any 
time  must  be  positive. 

3       Necessary  and  Sufficient  Conditions  for  Optimality 

In  this  section  we  discuss,  formally,  necessary  and  sufficient  conditions  for  a  consumption- 
investment  policy  to  be  optimal.  In  particular,  we  provide  a  verification  theorem  that  shows 
that  the  value  function  satisfies  a  differential  inequality.  Such  differential  inequalities  are  the 
hallmark  of  free  boundary  control  problems.  The  verification  theorem  requires  that  the  value 
function  be  twice  continuously  differentiable  in  W  ajid  once  continuously  differentiable  in  z  and 

y- 

In  general,  it  is  difficult  to  ascertain  the  smoothness  of  the  value  function.  In  Hindy  and 
Huang  (1993)  such  smoothness  could  be  established  because  there  is  a  closed-form  expression 
for  the  value  function.  In  the  problem  we  study  in  this  paper,  such  a  closed-form  solution  is 
not  currently  available.  For  this  reason,  the  numerical  analysis  relies  on  the  notion  of  viscosity 
solution  as  described  in  details  in  Hindy,  Huang,  and  Zhu  (1993).   We  record  the  verification 


*For  this  we  mean  both  the  Lebesgue  integral  and  the  Ito  integral  are  well-defined.  When  A(t)  is  a  feedback 
control  depending  on  (W{t),  z{t~),S(i),i)  and  C(t)  depends  on  the  history  of  (W,S),  we  mean  there  exists  a 
solution  W  to  the  stochastic  differential  equation  (5)  for  every  pair  of  controls. 


3     NECESSARY  AND  SUFFICIENT  CONDITIONS  FOR  OPTIMALITY  -  11 

theorems  for  smooth  value  functions  for  two  reasons.  Firsts  the  discussion  is  useful  for  conveying 
the  economic  intuition  of  the  optimal  solution.  Second,  the  verification  theorems  can  be-utilized 
in  the  future  if  a  closed-form  solution  is  obtained.  -  .   .. 

3.1      Necessary  Conditions 

We  use  Bellman's  optimality  principle  to  derive  necessary  conditions  about  J  assuming  that 
it  is  continuously  differentiable,  twice  in  W,  and  once  in  y  and  z.  One  difficiilty  that  arises 
is  that  the  usual  Bellman  equation  in  dynamic  programming  is  derived  when  the  admissible 
consumption  plans  are  purely  at  rates  and  are  feedback  controls.  Here,  cumulative  consumption 
can  be  in  gulps  and  may  have  singular  parts.  Moreover,  if  cumulative  consumption  has  singular 
components,  it  cannot  be  expressed  in  feedback  form.  Thus,  our  derivation  of  the  necessary 
conditions  wiU  be  heuristic  in  nature.  We  refer  the  reader  to  Zhu  (1991)  and  the  references 
therein  for  recent  work  on  the  class  of  singidar  control  problems. 
First,  we  observe  that 

(10)    JiW,  z,  y,  5)  =J{W-A,z  +  /?A,  y  +  AA,  5). 

if  a  consumption  gulp  of  size  A  is  prescribed  at  the  state  {W,z,y,S).  This  is  so  because  both 
quantities  are  equal  to  E[ji^  e~**u(z(t),  y{t))dt],  where  {z{t) ,  y{t);  s  €  [0,  oo)}  are  defined  along 
the  optimal  path  on  [0,oo).  Moreover,  the  size  of  the  gulp  should  be  chosen  to  maximize -J. 
Thus,  we  must  have 

JwiW-A,z  +  (3A,y  +  XA,S) 
^''>         =    f3J,{W-A,z-\-PA,y  +  XA,S)  +  XJy{W-A,z  +  PA,y  +  XA,S), 

where  JwiJz  a-nd  Jj,  denote  the  first  partial  derivatives  of  J  with  respect  to  its  first,  second, 
and  third  arguments,  respectively. 

We  now  show  that  (10)  and  (11)  imply  that  Jw  must  be  equal  to  PJ^  +  XJy  at  any  {W,z,y,S) 
where  a  gulp  of  consumption  is  prescribed.  Differentiating  (10)  with  respect  to  W  and  noticing 
that  A  is  an  implicit  function  of  W,  y,  and  z  defined  through  (11)  gives 

Jw{W,z,y,S) 
=    [-Jw{W  -A,z-\-PA,y  +  AA, S)  +  0MW  -A,z  +  0A,y  +  AA, S) 
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+XJy{W  -A,z  +  pA,y  +  AA, 5)]—  +  Jw{W  -  A, z  +  /JA,  5) 
=    7w(Vy-A,z  +  /?A,y  +  AA,5), 

where  we  used  (11)  to  get  the  second  equality.  Similarly,  we  have  JziW,z,y,S)  =  Jzi^W  — 
A,z  +  (3A,y  +  AA,  5)  and  Jy{W,  z,  y,  5)  =  Jy^W  -  A,  z  +  /3A,  y  +  AA,  5)  .  Given  (11),  we  then 
have  Jw{^,  Zi  y,  S)  =  ^Ji(Vr,  z,  y,  S)  +  \Jy{W,  z,  y,  5)  at  {W,  z,  y,  S)  where  a  consumption  gulp 
is  prescribed. 

Second,  assume  that  J  is  sufficiently  smooth  for  the  generalized  Ito's  lemma  to  applyJ  For 
any  time  t,  the  principle  of  optimality  in  dynamic  programming  and  the  generalized  Ito's  lemma 
imply  that 

0  =     max^cA  Et  [l/^^'  e-^'u{z{s),  y{s))ds  +  //+^*  e-^'[D^J{s)  -  SJ^s)]  ds 
(12)      +  //+^'  e-''[0Us)  +  XJyis)  -  Jw{s)]  dC{s)  +  ^i[J{r^)  "  J{ri)\  -  E.[^H'(r.)AVr(r.) 
+J,(r,)Az(r,)  +  Jy(r,)Ay(r,)]      a.s., 

where  dC  denotes  the  consumption  plan  on  the  time  interval  [f,f +  Ai),  r,  is  the  i-th  jump  point 
prescribed  by  dC  on  [f,  i+ A<),  V^J  is  the  differential  operator  associated  with  A  (see  Hindy  and 
Huang  (1993,  footnote  8),  J{s)  and  its  derivatives  are  evaluated  at  {W{s),z{s~),y{s~),S{s)), 
and  we  have  assumed,  without  loss  of  generality,  that  Ito  integral  appearing  in  Ito's  lemma  is 
a  martingale  and  thus  vanishes  when  the  conditional  expectation  is  taken. 

Since  no  consumption  is  always  feasible,  putting  dC  =  0  in  (12)  and  letting  At  decrease  to 
zero  gives 

(13)    0>  u(z,y)  +  max[P'*J] -<5J. 

A 

This  relation  must  hold  for  all  values  of  W,  y,  z,and  S. 

Suppose  t  is  not  a  point  for  consumption  gulps.  Then  for  small  enough  A,  by  right- 
continuity  of  C,  there  will  not  be  any  gulps  on  [t,t  +  At).  For  a  nontrivial  consumption  dC, 
with  no  discontinuities  on  [t^t  +  Af),  to  maximize  the  right-hand  side  of  (12)  for  small  enough 
At,  it  must  be  the  case  that  Jw{t)  =  PJzit)  +  AJy(f).  This,  together  with  earlier  discussion 


^The  generalized   Ito  lemma  we  will  use  throughout  can  be  found  in  Krylov  (1980,   theorem  2.10.1)  and 
Dellacherie  and  Meyer  (1982,  VIII.27). 
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when  there  is  a  consumption  gulp  at  t,  implies  that  nontrivial  consumption,  independent  of  its 
form,  can  only  occur  at  time  t  when 

0MW{t),  z{t-),  y(r ),  S{t))  +  xjy{W{t),  z(r ),  y(r),  S{t)) 
=    Jw{W{t),z{r),y{r),S{t)). 

Moreover,  when  optimal  cumulative  consumption  is  continuous  on  [t,t  +  At),  standard  argu- 
ments in  dynamic  programming  show  that  (13)  holds  as  an  equality  at  t  when  At  decreases  to 
zero. 

Next  suppose  that  the  optimal  consumption  plan  prescribes  zero  consumption  on  [t,t  +  At). 
For  zero  dC  to  maximize  the  right-hand  side  of  (12)  for  small  enough  At,  it  must  be  the  case 
that 

PMWit),  z{t-),  y(r ),  5(0)  +  xpj.iw{t),  z{t-),  y(r),  s{t)) 
<Jw{W{t),z{t-),y{n,S{t)). 

Moreover,  given  that  dC  =  0  is  the  optimal  consumption  plan  at  t,  (13)  holds  as  an  equality. 
In  summary,  we  have  derived  the  following  necessary  conditions  for  optimality: 

•  consumption  gulp  at  i: 

l3J,{t)  +  XJy{t)  -  Jwit)  =  0        u{z{t),  y{t))  +  maxfP-*  J(0]  -  SJ{t)  <  0 

A 

•  continuous  consumption  at  t: 

(3J,{t)  +  \Jy{t)  -  Jw{t)  =  0        u{z{t), y{t))  +  max[X>^7(0]  -  6J{t)  =  0 

A 

•  no  consumption  at  t: 

f3J,{t)  +  XJy{t)  -  Jw{t)  <  0        ri{z{t),  y{t))  +  max[P^  J(0]  -  SJ{t)  =  0 

A 

These  conditions  can  be  written  compactly  as  a  differential  inequality: 
(14)    max|max[u(r,y)-|-P^J-^J],  flJ^  +  XJy-Jw\  =  0, 
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plus  the  condition  that  nontrivial  consumption  occurs  only  at  times  t  where  (iJz(t)  +  AJj,(<)  = 
Jw{t). 

In  addition  to  the  above  necessary  conditions,  it  is  clear  that  J  must  satisfy  the  following 
boundary  coaditioa: 


(15)    ^mJ{W,z,y,S)=   H e-'U{ze-P\ye-'^')dt . 
vvio  Jo 


Condition  (15)  is  implied  by  the  constraint  that  wealth  at  any  time  cannot  become  negative. 
Thus,  whenever  wealth  is  zero,  the  only  feasible  policy  afterwards  is  no  consumption. 

3.2      Sufficient  Conditions 

In  this  section  we  provide  a  verification  theorem  to  check  the  optimality  of  investment-consumption 
plans.  We  show  that  if  there  exists  a  solution  J  to  the  differential  inequality  (14)  with  the  bound- 
ary condition  (15),  and  J  satisfies  some  regularity  conditions,  then  J{W,  2,  y,  S)  >  J{W,  z,  y,  S) 
for  all  {W,  z,  y,  S).  We  then  give  conditions  so  that  J{W,  z,  y,  S)  is  attained  by  a  candidate 
feasible  investment  and  consumption  policy.  It  then  follows  that  J{W,  z,  y,  S)  =  J{W,  z,  y,  S) 
and  the  candidate  investment  and  consumption  policy  is  the  optimal  policy. 

Proposition  1  Let  J  :  If^"*"^  —*■  5J+  be  positive,  concave  in  its  first  three  arguments,  contin- 
uously differentiahle  over  Hir^'^'^  in  all  of  its  arguments,  and  twice  continuously  differentiahle 
over  Ht^"^^  in  its  first  N  ■\-'i  arguments,  except  possibly  on  a  smooth  manifold  M.^  satisfying 
the  differential  inequality  of  (I4)  with  the  boundary  condition 


lim  J{W,z,y,S)=  f°° e-^^u{ze-^\ye-^')dt  <  00. 


In  addition,  J  satisfies  the  growth  condition:  for  every  T  G  5J+,   there  exists  KJ  >  0  and 
Kj  >  0  so  that 

(16)    \JiX)\<KUl  +  \X\f^     WX  =  iW,z,y,S)eR':^^\ 

Assume  furthermore  that  there  exists  a  consumption  policy  C  £  C  financed  by  A"  G  A,  with 
the  associated  state  variables  W',y*,  and  z' ,  such  that,  putting  g  =  inf{<  >  0  :  W{t)  =  0}, 


'For  the  definition  of  smooth  manifolds  see  Hindy  and  Huang  (1993,  footnote  8). 
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Vt  e  (0,^],  P-a.5. 

(17)  -SJ  +  V^'J  +  u{z',y')     =     0 

(18)  f\jwiW\z-,y',S)-pj\{W',z-,y',S)-XJ,iW',z',y',S)]dC'{s)    =     0 

(19)  ]imE[e-''J{W'{t),z'{t-),y'{r),S-it))]     =     0, 

(— ►oo 

and,  almost  surely, 

(20)  J{W{Ulz{t-),y{t-),Sit,)) 

=  JiWiU)  -  AC'it,),  z{t-)  +  PAC'iU),  yit-)  +  XAC'iU),  5(1,)) , 

where  ti's  are  the  times  of  gulps  prescribed  by  C  on  [0,^).    Then  J  =  J  and  {C',A')  is  an 
optimal  consumption  and  investment  policy. 

Proof.  The  proof  is  similar  to  that  of  Proposition  1  and  Theorem  1  in  Hindy  and  Huang 
(1993)      I 

Note  that  Proposition  1  requires  two  conditions.  First,  J  is  positive.  Second,  the  expected 
value  of  J{t)  along  the  optimal  path  must  converge  to  zero  as  t  increases  to  infinity.  The 
latter  condition  ensures  that  the  agent  exhibits  enough  impatience  so  that  accumulating  wealth 
indefinitely  without  consumption  is  not  optimzd.  We  imposed  the  former  condition  for  technical 
convenience  and  we  can  replace  it  by  a  stronger  condition  which  requires  that  (19)  holds  not 
just  for  the  optimal  policy  but  for  all  feasible  plans. 

As  a  recipe  for  decision  making.  Proposition  1  outlines  general  principles  that  the  agent 
should  follow  as  long  as  his  wealth  is  strictly  positive.  In  particular,  the  theorem  instructs  the 
agent  to  use  a  control  policy  that  keeps  the  triple  {W,  z,  y)  in  the  region  where  the  differential 
equation  (17)  is  satisfied.  In  addition,  the  agent  may  consume  only  when  marginal  utility  of 
wecdth  is  equal  to  the  sum  of  (/?  times)  the  marginal  utility  of  the  service  flow  and  (A  times) 
the  marginal  utility  of  the  standard  of  living.  Finally,  in  the  contingency  that  the  optimal 
consumption  policy  calls  for  a  "gulp",  the  size  of  the  gulp  should  be  chosen  such  that  the  value 
function  immediately  before  is  equal  to  the  value  function  immediately  after  the  gulp. 
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The  optimal  consumption  and  portfolio  policy  characterized  in  Proposition  1  of  course 
satisfies  all  the  necessary  conditions  of  Section  3.1.  Applying  Proposition  1  requires  that  one 
know  a  priori  that  the  value  function  is  twice  continuously  differentiable.  Absent  this  knowledge, 
we  can  use  a  weaker  notion  of  solution  to  the  differential  inequality  (14).  In  Hindy,  Huang,  and 
Zhu  (1993),  we  use  the  notion  of  viscosity  solutions,  Crandall  and  Lions  (1982),  and  show  that 
such  a  solution  of  (14)  is  indeed  the  value  function  of  the  optimal  consumption  problem. 

4      Numerical  Approach 

In  this  section  we  describe  the  numerical  approach  utilized  to  solve  the  infinite  horizon  utility 
maximization  problem.  We  replace  the  continuous  time  processes  {W,  z,  y)  by  a  sequence  of 
approximating  discrete  parameter  Markov  chains.  The  original  optimization  problem  is  then 
approximated  by  a  sequence  of  discrete  parameter  control  problems.  We  show  that  the  value 
function  of  each  Markov  chain  control  problem  satisfies  an  iterative,  and  hence  easily  pro- 
grammable, discrete  BeUman  equation.  Bellman's  equation  of  the  discrete  control  problem  is 
a  natural  finite-difference  approximation  to  the  continuous  Bellman's  'nordinear'  partial  differ- 
ential equation.  We  also  show  that  the  method  of  Markov  chain  approximation  produces  a 
consistent,  stable,  and  convergent  numerical  scheme.  Furthermore,  the  special  monotonicity 
and  concavity  features  of  the  problem  allow  us  to  prove  convergence  npt  ordy  of  the  discrete 
approximations  of  the  value  function  J{W^  y,  z),  but  also  of  the  corresponding  discrete  approx- 
imations of  the  optimal  policies,  when  the  latter  exist. 

For  numerical  implementation,  we  need  to  restrict  the  domain  of  definition  of  the  state 
variables  to  a  finite  cubic  region  [0,  Mw]  x  [0,  My]  X  [0,  M^].  For  this  purpose,  we  introduce  the 
reflection  processes  Lt,  Rt  and  Dt  at  the  boundaries  of  the  cubic  region.  We  hence  consider  the 
following  modified  state  variables: 

dyt     =     —Xytdt  -\-  XdCt  —  dLt , 
dzt     =     -fiztdt  -I-  lidCt  -  dRt ,     and 
dWt    =     Wt{r  +  Atifi  -  r))dt  -  dCt  +  WtAtadBt  -  dDt , 

where  Lf,  Rt  and  Dt  are  nondecreasing  and  increase  only  when  the  state  processes  (W,z,y)  hit 
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the  cutoff  boundaries  W  =  Mw,  z  =  M^,  and  y  =  My,  respectively.  - 

We  will  assume  that  the  riskless  rate  r  is  constant  and  that  there  is  a  single  risky  asset 
whose  price  process  is  a  geometric  Brownian  Motion  given  as 


S{t)+  f  p{s)ds  =  5(0)+  I  fMS{s)ds+  I  aS{s)dB{s), 
Jo  Jo  Jo 


where  n  >  r  and  a  are  positive  constants  and  where  B{t)  is  a  standard  Brownian  Motion. 
Since  the  investment  environment  is  stationary,  the  value  function  J  depends  only  on  the  level 
of  wealth  W,  the  stock  of  the  durable  good  z,  the  standard  of  living  y,  and  time  t.  It  is  also 
easy  to  see  that  J{W,  z,  y,  t)  =  e~  J{W,  z,  y,  0)  since  the  felicity  function  is  time  separable  with 
a  constant  impatience  parameter.  We  will  focus  our  analysis  on  the  function  J{W,  z,  y,  0)  and 
denote  it  henceforth  by  J{W,  z,  y). 

We  will  solve  the  following  infinite  horizon  program  on  the  restricted  domain: 


(21)    Max         E         e-^'uiyt,zt)dt  =  J^{W,y,z) 
Jo 


subject  to  the  dynamics  of  z  and  y  and  the  dynamic  budget  constraint.  Bellman's  equation  for 
this  dynamic  program  takes  the  form: 

(22)    max{-(5 J^  +  max{£^ J^}  +  u(y,  z),  XJ^  +  pj^  -  J^}  =  0, 

on  [0,  Afvv]  X  [0,A/y]  x  [0,  A/^],  together  with  the  appropriate  boundary  conditions,  where  the 
operator  L^  is  given  by  £^  =  \W^A'^ct^^  +  W[r  +  A{fi  -  r)]^  -  Xy^  -  /3z^. 
We  follow  the  lead  of  Soner  (1986)  and  add  to  (22)  the  boundary  constraints 

QjM  QjM  QjM 

__  =  0,  —  =  0,     and         —  =  0, 

respectively,  at  the  cutoff  boundaries  W  =  Mw-,  y  =  My,  and  z  =  Mj.  These  constraints 
reflect  the  asymptotic  behavior  of  the  value  function  on  the  original  unbounded  domain.  It  is 
worthwhile  to  remark  that  in  the  theoretical  analysis  of  convergence  of  the  numerical  scheme, 
the  value  functions  J^  converge  pointwise  to  7  as  the  sequence  of  restricted  domains  Q\f 
increases  to  Q,  regardless  o/ the  specification  of  the  behavior  of  7^  at  the  boundaries.  The 
boundary  conditions  we  chose,  however,  affect  the  quality  of  the  actual  numerical  solution. 
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4.1      The  Markov  Chain  Scheme 

We  introduce  a  sequence  of  grids 

Gh  =  {ik,iJ):W  =  kxh,y  =  ixh,z  =  jxh;0<k<  Nw,0  <  i<  Ny,0  <  j  <  N^}  , 

where  h  is  the  step  size,  and  where  Ny  =  My/h,  N^  =  M^/h,  and  Nw  =  Mw /h  are  integers. 
Each  grid  point  (fc,  i,j)  €  Gh  corresponds  to  a  state  {W,  j/,  z)  with  W  =  kxh,  y  =  ixh,  and 
z  =  j  X  h.  For  simplicity,  we  take  the  same  step  size  in  all  three  directions.  For  a  fixed  grid, 
we  introduce  the  space  of  discrete  strategies 

ACn  =  {(A,C);A  =  /x  A,AC  =  OorA;0  </  <  Na} 

where  A  is  the  step  size  of  control,  N^  =  A/ A  is  total  number  of  control  steps  with  A  an 
artificial  bound  to  be  relaxed  in  the  limit  as  A  i  0  and  Na  x  A  t  oo- 

We  approximate  the  continuous  time  process  {W,  z,  y)  by  a  sequence  of  discrete  parameter 
Markov  chains  {{Wl^,y^,z^);n  =  1,2,  •••}  with  h  denoting  the  granularity  of  the  grid  and 
n  denoting  the  steps  of  the  Markov  chain.  For  every  h,  the  chain  has  the  property  that,  at 
each  step  n,  there  is  a  choice  between  investment  and  consumption.  At  any  time,  a  chain  can 
either  make  an  instantaneous  jump,  (AC  =  h),  or  follow  a  "random  walk",  (AC  =  0),  to  the 
neighboring  states  on  the  grid.  At  the  cutoff  boundaries  the  chain  is  reflected  to  the  interior 
in  a  manner  consistent  with  its  dynamics  in  the  interior  of  the  domain.  Fix  a  chain  and  its 
corresponding  grid  size  h.  The  transition  probabilities  for  this  Markov  chain  are  specified  as 
follows: 

1.   The  case  of  no  consumption — (AC  =  0): 

The  chain  can  possibly  move  from  the  current  state  {k,  i,j)  only  to  one  of  the  four  neighboring 
states:  {k  +  l,i,j),{k-l,i,j),  {k,i-  l,j),and  {k,i,j -I).  For  any  investment  policy  A  =  /x  A, 
the  transition  probabilities  in  this  case  are  defined  as 


pak,i.JUk-ui,j)]   =   ^^^-^ 
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P;^[ik,iJ),{k,i-hj)]    = 


QHk,ijy 


Pi^[{k,i,j),{k,i,j-l)]     =     Q^^y     and 

Pl^[{k,i,j),{k,i,j)]    =     l-Pi^[{k,iJ),{k+l,i,j)]-PI^[ik,iJ),ik-\,i,j)] 

-P(^[{k,i,j),ik,i-l,j)]-P^[(k,i,j),ik,i,j-l)] 

where  the  normalization  factor  Q^{k,i,j)  is  given  by 

Q''{k,i,j)  =    max  [k^h\l  X  A)V  +  kh'^[r  +  lxA{n  +  r)]  +  (At  +  f3j)h^]. 

The  recipe  for  these  transition  probabilities  is  a  slightly  modified  version  of  those  suggested  by 
Kushner  (1977). 

Furthermore,  we  define  the  incremental  difference  Ay^  =  y^^i  -  y^,  Az^  =  z^^^  —  2^,  and 
AW^  =  Ty^^.1  —  W^.  At  step  n  of  the  chain,  with  the  previously  defined  time  scale  A„f'',  one 
can  verify  that 

i:„^[Ay^]  =  -AyAt^-AL^ 

,23N  ^n[^^n\  =  -PzAt),-ARi, 

^  ^/^[AH^n']  =  M^[r  +  A(/i-r)]At^-AZ)^  and 

i;^[AH^„^  -  £;^[AVy„^]]2  =  V^2^V2At^  +  C»(A<i), 

where  E^  denotes  expectation  conditional  on  the  nth-time  state  (H^,J,  y^,  2^),  and  where  the 
reflecting  processes  AX^,  Ai2j,  and  AjD^  equal  to  the  positive  value  h  only  when  y^,  2^,  and 
W^  reach  their  respective  boundaries.  This  implies  that  the  first  and  second  moments  of  the 
Markov  chain  approximate  those  of  the  continuous  process  {Wt,y,,Zi).  We  call  this  property 
"local"  consistency  of  the  Markov  chain. 
2.   The  case  of  consumption —  (AC  =  h): 

The  chain  jumps  along  the  direction  (— l,A,/3)  from  the  current  state  {k,i,j)  to  the  state 
(fc  -  1,  i  +  A,  J  +  /3).  However,  the  later  state,  {k  —  1,  t  +  A,  j  +  /?),  is  usually  not  on  the  grid 
except  in  the  trivial  case  A  =  1  =  /?.  For  ease  of  programming,  we  take  the  intersection  of  the 
direction  vector  (  — l,A,/3)  with  the  corresponding  surface  boundary  and  randomize  between 
three  grid  points  adjacent  to  the  intersection  point.    We  randomize  in  such  a  way  that  the 
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expected  random  increment  will  be  along  the  direction  (  — l,A,/3)  and  of  length^equal  to  the 
distance  from  the  starting  state  to  the  point  of  intersection. 

Without  loss  of  generality,  we  assume  hereafter  that  A  <  1  <  /3.  In  this  case,  the  intersection 
occurs  inside  a  triangle  spanned  by  the  three  grid  points  {k,i,j  +  1),  (k  —  l,i,j  +  1),  and 
{k  -  l,i  +  l,j  +  I).  The  transition  probabilities  can  be  defined  as: 

P^[{k,i,j),{k,iJ+l)]     =     ^, 

Pfp,z,j),(fc-l,t,j+l)]     =     ^,  and 

Pf[(fc,i,j),(A:-l,i+l,i+l)]     =     ^, 
In  this  case,  also,  we  can  verify  the  property  of  "local"  consistency: 

(24)    £^[Ay^]  =  AAC,         £„^[Az^]  = /3AC,         and         E^^lAW!:]  =  -  AC 

where  AC  =  3  is  the  increment  of  consumption.  These  quantities  correspond  to  the  respective 
changes  in  the  continuous  time  process  for  a  consumption  increment  of  the  same  magnitude 
AC.  The  movements  of  the  Markov  chain  are  depicted  in  Figure  1. 

4.2     The  Markov— Chain  Decision  Problem 

A  policy  {A,C)  is  admissible  if  it  preserves  the  Markov  property  in  that 

conditional  on       !;V^.'^J"/<'.         /       \  Pk[(f^^i^J)dk\i',j%   if  AC  =  h  , 


where  W  =  kh  and  W  =  k'h,  y  =  ih  and  y'  =  i'h,  z  =  j7i  and  z'  =  j'h.  The  control  problem 
for  the  discrete  parameter  Markov  chain  is  then  to  solve  the  program: 

(25)    7''(fc,t-,i)  =  inax  Et..  f]  ^"""«(3/n,4)^'n, 

n=0 

where  f^  =  IZo<Kn  ^^''  and  Af''(Jb,  t,j)  =  h} IQ^{k,i,j).  Note,  this  is  analogous  to  (21)  in  the 
sense  that  the  sum  in  (25)  approximates  the  expected  integral  in  (21). 


4     NUMERICAL  APPROACH  21 

The  discrete  dynamic  programming  equation  now  takes  the  iterative  form  •  ^      r-: 

J>^{k,iJ)  .=     m^^{Zk;,',:'P^[{k,i,j),{k\i',j')]JHk',i',j'),  .        . 

(26)  maxo<,<N.  {e-*^'''^*''-^)  Eit',-,/  P^^Kk,  i,j),  {k',  i',j')]J'{k',  i\j')] 

+  UiiJ)At\k,i,j)] 

for  {k,i,j)  6  Gn,  with  iterative  reflection  at  the  artificial  boundaries,  where  we  recall  that 
P^[*,*]  and  P^[*y*]  are  the  transition  probabilities  of  the  chain.  Now,  let  us  denote 

D-  Klc  i  i\  -  Ak,iJ)-Jik,i-l,j)  _  J{k,i,j)-J{k,i,j-l) 

n+  j(L.  ,•  ,•>.  _  J{k,i+l,j)-J{k,i,j)             +     ■            _  Jik,i,j+l)-J(k,i,j) 
V-  J{k,t,j)= ,        Dj  J{k,i,j)  - , 

n+r/L   •    -x       J{k  +  l,i,j)-J{k,i,j)                        .            J{k,i,j)-J{k-l,i,j) 
DJJik,  J,  j)  = ,        D^  J{k,  t,  j)  = ,  and 

r,2  jri.  ■    x      J(k  +  l,i,j)-2J{k,i,j)  +  J{k  -  l,i,j) 

^k  •/(,«,»,;)  = j^ • 

Using  this  notation,  we  can  express  the  discrete  Bellman's  equation  (26)  in  the  form: 

^  _g-5A«''(fc,.J)^ 
"   ^      At''(fc,i,j) 

AZ?t7''(fc-l,i,j+l)  +  /3Z)t7''(fc,i,j)-D^j''(fc,i,j+l)    <    0,  with 
£^  =  hv'^A^o'^Dl  +  W^(r  +  AyL)Dt  -  WAtDI  -  XyD;  -  fizDj. 

Furthermore,  one  of  these  two  inequalities  must  hold  as  an  equality  at  each  (k,i,j)  €  G^. 

Clearly,  (1  —  e~^^'  )/^^^  approximates  the  discount  factor  A  as  /i  — ►  0.  As  a  result, 
the  above  discrete  differential  inequalities  are  a  finite-diflference  approximation  of  Bellman's 
equation  (22)  for  the  majcimization  problem  on  the  finite  domain  [0,Afw]  x  [0,Afy]  X  [0,Mj]. 
At  the  cutoff  boundaries,  we  specify  a  slightly  different  finite-difference  approximation.  We 
refer  the  reader  to  Hindy,  Huang,  and  Zhu  (1993)  for  the  details. 

4.3     Convergence  of  the  Markov  chain  Approximation 

In  this  section,  we  state,  without  proofs,  the  convergence  results  of  the  Markov  chain  approxi- 
mations. Specifically,  we  state  that  the  value  functions  in  the  sequence  of  Markov  chain  control 


J'  +  max{CtJ\k,i,j)}  +  UiiJ)    <     0, 
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problems  converge  to  the  value  function  of  the  continuous  time  problem  on  the  bounded  do- 
m£un.  Furthermore,  the  optimal  portfolio  policies  and  the  optimal  consumption  regions  in  the 
sequence  of  the  Markov  chain  decision  problems  converge  to  those  of  the  continuous  time  prob- 
lem, if  the  latter  exist.  For  an  extensive  discussion  of  the  sense  of  convergence  and  for  the 
technical  details  and  proofs,  we  refer  the  reader  to  Hindy,  Huang,  and  Zhu  (1993). 

Theorem  1  The  discrete  Markov  chain  approximation  yields  a  consistent  and  stable  finite- 
difference  scheme  for  Bellman's  equation  (22).  Consider  a  sequence  of  grids  and  let  the  grid 
size  h  I  0  such  that  (hk,  hi,  hj)  — ►  {W,  z,  y).  The  sequence  of  solutions  J^{k,  i,j)  of  the  discrete 
Bellman's  equation  converges  to  the  value  function  J{W,  y,  z)  of  continuous-time  problem  as 
h  [  0.  Furthermore,  if  there  exists  an  optimal  policy  {A',C')  for  the  original  problem,  then 
as  h  I  0,  the  sequence  of  approximate  policies  A^{k,i,j)  converges  to  the  optimal  investment 
policy  A'{W,z,y).  Finally,  if  the  point  {W,z,y)  is  in  the  optimal  consumption  (abstinence) 
region  in  the  original  problem,  then  almost  all  grid  points  that  converge  to  {W,  z,  y)  will  be  in 
the  consumption  (abstinence)  regions  of  the  corresponding  Markov-chain  problems. 

5      Numerical  Implementation 

In  this  section,  we  describe  briefly  the  implementation  of  the  numerical  scheme  for  solving 
the  discrete  Markov  chain  program.  We  designed  the  numerical  scheme  to  reflect  the  iterative 
nature  of  the  discrete  Bellman's  equation: 

J'ik,ij)    =    mB^{^,.^i,j,P^[{k,i,JUk',i'J')]j\k',i',j') 

+    Tn^xo<i<NA^-^^''j:k',,-,j'PH[{k,iJ),ik',i'J')]j'^{k,i,j)}+Uii,j)At}. 

At  each  state,  we  first  need  to  choose  between  consumption  (AC  =  h)  and  no  consumption 
(AC  =  0).  If  consumption  is  chosen,  we  use  the  consumption  scheme 

J\k,ij)=  J^  P^[{k,i,jUk',i',j')]j\k',i'j'). 

Otherwise,  we  use  the  investment  scheme 

J''(fc,t,j)=    max   e-^^'   ^   P;^[{k,iJUk',i',j')]j\k,i,j)^U{i,j)At 
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where /'^[♦,  *]  and  PJp[*,  *]  are  given  in  section  4.  — - 

The  numerical  algorithm  is  a  combination  of  two  iterative  schemes:  approximation  in  policy 

space  and  approximation  in  value  space.  The  first  method  can  be  viewed  as  a  "descent"  method 

in  the  space  of  control  policies,  while  the  second  method  calculates  the  (n+l)-step  value  function 

with  the  updated  policies  from  the  previous  iteration. 
Specifically,  we  implement  the  following  steps. 

1-  Guess  an  initial  value  function  Jo{k,i,j)  =  K  x  U{i,j)  and  set  the  initial  policies  as 
Ao(fc,t,j)  =  0  =  ACo(fc,t,i)  for  {k,i,j)  6  Gh  where  /if  >  0  is  a  constant,  and  U{i,j)  =  u{y,z) 
is  simply  the  utility  function.  ''■ 

2-  Given  the  n-step  value  function  {J^{k,  i,j) :  {k,  i,j)  G  Gh}  and  the  n-step  policies  {A^(fc,  i,  j), 
ACJJ(A;,t,  J )},  compute  at  each  state  (fc,t,j)  €  Gh  the  updated  policies  {A!|^^■^{k,i,j),AC^^■^{k,i,j)}. 
At  each  {k,i,j)  6  Gh,  compute  the  maximum  attainable  utility  from  investment  as 

max^e-^^'{p,^[(fc,z\j),(fc+l,i\j)]JiJ(fc+l,t\i)+P,^[(fc,i,i),(/:-l,i,j)]J„^(fc-l,f,j) 
+Pl^[{k,i,JUk,i-l,j)]Jil{k,i-l,j)  +  P^[{k,i,j),ik,i,j-l)]j:i{k,i,j-l) 

+P^[{k,i,j),{k,i,j)]j!lik,iJ)}+Uii,j)At\ 

where  Af''  is  the  interpolation  time  and  P'*[*,  *]'s  are  the  transition  probabilities  of  the  chain. 
Meanwhile,  we  also  compute  the  utility  for  consumption  by 

^jl^{k,ij+i)  +  ^j![{k-i,i,j  +  i)  +  ^j!^{k-i,i+i,j  +  i) 

which  equates  the  marginal  utility  of  we3dth  to  a  combination  of  the  marginal  utility  of  con- 
sumption and  the  marginal  utility  of  living  standard.  Then,  update  the  (n  +  l)-step  policies 
by  choosing 

A>:,^,{k,i,j)  €  argmax^=;,^,o</<;.^{e-'^'''    E   P^[ik,iJ),{k',i'J')]J::{k',i'J')}, 

and  AC^^i(fc,i,  j)  =  0  if  the  utility  from  investing  is  higher  than  that  from  consuming.  If  the 
reverse  is  true,  we  choose  AC^^i(fc,i,j)  =  h  and  An^i(k,i,j)  is  set  to  some  arbitrary  number. 

3-  With  the  updated  policy  (>l^+i,  AC^+j),  we  now  evjJuate  the  utility  functional  to  obtain  the 
(n  +  l)-step  value  function  J^^i-  In  this  step,  first  update  the  transition  probabilities  P-^[*,*] 
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according  to  the  new  policies.- Then  at  each  {k,i,j)  6  G°,  compute  Jn+ii^^hj)  by  solving  a 
linear  system: 

+P/f[(fc,i,j),(/:-l,i,i)]J„\i(*-l,t,i) 
+PI^[{k,iJ),ik,i-  l,j)]J„Vi(fc,»-  l,i) 
+P,^p,»,i),(fc,t,j-  l)]J„V(fc,t,i-  1) 
+P/^[{k,i,j),ik,i,j)]j!^^,{k,i,j)} 
+  U{i,j)At\ 

where  At'^  =^'/Q""^^(fc,i,j),  and  Q^'^^{k,i,j)  is  the  normalizing  constant  as  in  the  previous 
step  but  with  the  (n  +  l)-step  updated  policies  An+i{k,i,j).  On  the  other  hand,  if  consumption 
is  prescribed  at  {k,i,j)  £  G°  (i.e.  AC^^i(fc,i,j)  =  h),  compute  Jn+i{k,i,j)  by  solving  the 
following  different  linear  system: 

+  ^j!^+,{k-l,i+l,j  +  l) 

Finally,  the  evaluation  at  the  cutoff-boundary  can  be  obtained  according  to  the  reflection  rules 
specified  in  Hindy,  Huang,  and  Zhu  (1993,  §5.4).  Throughout  the  entire  procedure,  the  value 
at  A;  =  0  is  chosen  to  satisfy  the  boundary  condition  at  W^  =  0. 
4-  Compute  the  roundoff  error 


1 


If  ERR{n)  <  €,  the  desired  precision  is  achieved  and  the  program  is  terminated  with  the 
results  taken  bls  the  value  j'^{k,i,j)  a.nd  the  optimal  policies  {A'^{k,i,j),C'*{k,i,j)}.  Otherwise, 
increase  n  by  1  and  return  to  steps  2  and  3.  Here,  e  >  0  is  the  tolerance  error  prescribed  in  the 
program. 

Computing  the  value  function  at  every  iteration  requires  solving  a  large  linear  system  of 
equations.  We  use  a  relaxation  method  to  solve  this  system  rather  than  a  method  that  requires 
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computing  the  inverse  of  a  matrix.  The  large  size  of  the  involved  three  dimensional  matrices 
renders  the  latter  algorithm  numerically  very  expensive.  The  usage  of  the  relaxation  technique 
is  guaranteed  to  produce  accurate  results  because  of  the  fact  that  the  matrices  describing  the 
linear  system  to  be  solved  define  a  contraction  mapping  as  explained  in  Hindy,  Huang,  and  Zhu 
(1993,  §6.1).  The  numerical  results  are  discussed  in  the  following  section. 

6      Numerical  Solution 

In  this  section,  we  report  the  optimal  consumption  and  portfolio  rules  in  an  infinite  horizon 
program  when  the  utility  function  is  u{z,y,t)  =  e~*'z'*'y"^,  with  0  <  a,-  <  1,  t  =  1,2.  The 
discount  factor  6  expresses  the  impatience  of  the  agent.  The  differential  inequality  (14)  simplifies 
to 

(27)  max  Iz^'y"'  +  max[P^J]  -  6 J,  pj,  +  \Jy  -  Jw\  =  0. 
The  boundary  condition  when  W  =  0  is 

(28)  J(0,.,v)=^^^^;^^^. 

We  solve  this  problem  numerically  and  restrict  the  domain  of  the  state  variables  {W,z,y) 
to  the  cube  [0, 1]  x  [0, 1]  X  [0, 1].  Observe  that  the  multiplicative  specification  of  the  utility 
function  is  well  defined  for  all  strictly  positive  values  of  z  and  y.  Furthermore,  the  utility 
function  is  increasing  in  both  z  and  y.  A  similar  multiplicative  specification  was  introduced  by 
Abel  (1990).  The  life  time  satisfaction  of  the  agent  increases  as  the  standard  of  living  increases, 
ceteris  paribus.  This  contrasts  with  the  "difference  specification"  studied  by  Constantinides 
(1990)  and  Sundaresan  (1989)  who  analyze  the  form  u{z,y,t)  =  e~*'(z  —  yy,  for  some  7  <  1. 
This  utility  function  is  only  well  defined  when  z  >  y.  For  2  <  y,  we  can  take  u{z,y,t)  =  —00. 
This  extension  maintains  the  concavity  of  the  utility  function. 

Constantinides  (1990)  and  Sundaresan  (1989)  study  the  special  case  when  /3  =  00  and  hence 
z{t)  is  the  consumption  rate  c{t)  at  time  t.  The  difference  utility  function  implies  that  the  life 
time  satisfaction  of  an  agent  is  lower,  the  higher  is  his  standard  of  living.  Furthermore,  this 
specification  requires  that  initial  wealth  Wq  be  at  least  equcd  to  zo/r  for  the  life  time  utility 
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mjLximJzation  problem  to  have  a  feasible  policy  that  keeps -the  consumption  rate  higher  than 
the  standard  of  living  at  all  future  dates.  An  agent  who  starts  with  Wq  <  zq/t  is  certain  to  fail 
to  maintain  the  initial  standard  of  living.  As  a  result,  the  maximum  life  time  utility  of  such 
an  agent  is  — oo.  For  interest  rates  in  the  range  2%  —  20%,  an  agent  requires  5  —  50  times  the 
initial  standard  of  living  in  financial  wealth  to  avoid  the  extremely  painful  consequences  of  the 
difference  specification.  This  requirement  restricts  the  domain  of  the  state  space  where  the  life 
time  utility  optimization  problem  is  meaningful. 

Before  we  proceed,  we  observe  that  the  felicity  function  u{z,y,i)  is  homogeneous  of  degree 
Qi  +  02  in  2  and  y.  In  addition,  the  dynamics  of  the  state  variables  W,  z,  and  y,  given  in 
(5),  (6),  and  (7),  respectively,  are  linear.  As  a  result,  the  value  function  J  is  homogeneous 
of  degree  qi  +  qj.  In  other  words,  J{kW,kz,ky)  =  k°^'^°''J{W,z,y)  for  all  positive  k.  In 
principle,  we  could  use  the  homogeneity  of  the  value  function  to  reduce  the  number  of  state 
variables  from  three  to  two.  For  example,  we  could  divide  by  the  stock  level  z  and  write 
J{W,z,y)  =  z°'^'^°^J{W/z,l,y/z).  We  could  then  deal  with  the  new  state  variables  xi  =  ^ 
and  X2  =  ^  and  rewrite  Bellman's  equation  in  terms  of  xi  and  X2- 

A  reformulated  problem  with  two  state  variables,  however,  is  not  easier  to  analyze  com- 
putationally. Let  the  value  function  in  the  reformulated  problem  be  z°'^'^°''V{xi,X2).  The 
optimality  condition  Jw  —  0Jz  —  ^Jy  in  the  original  formulation  corresponds  to  (1  +  Pxi)Vx^  + 
{Px2  —  ^)Vxj  —  /3{ai  +  a2)V  in  the  new  formulation.  Note  that  in  the'  new  formulation,  this 
condition  is  expressed  as  a  linear  partial  differential  equation  with  zero-order  term  /?(ai  -\-a2)V. 
As  a  result,  the  numerical  solution  of  this  equation  is  susceptible  to  numerical  instabilities  that 
require  very  small  grid  size  or  the  use  of  implicit  techniques.  For  more  on  the  stability  of  nu- 
merical schemes,  we  refer  the  reader  to  Celia  and  Gray  (1992,  §4.2).  The  same  condition  in  the 
original  formulation  does  not  contain  any  zero-order  terms  and  hence  is  immune  to  numerical 
instability. 

There  is,  hence,  a  tradeoff  in  choosing  one  of  the  two  theoretically  equivalent  formulations 
of  the  problem.  On  the  one  hand,  the  three  dimensional  formulation  requires  more  computer 
memory  to  process  the  three  dimensional  grid.  The  numerical  algorithm,  however,  is  simple  and 
free  of  numerical  instability.  On  the  other  hand,  the  two  dimensional  formulation  demands  less 
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storage  because  of  the  reduction  in  the  number  of  state  variables.  This^'formulation,- however, 
requires  special  techniques  to  handle  numerical  instability.  Such  techniques  reduce  the  speed  of 
convergence  of  the  algorithm.  We  elected  to  solve  the  problem  numerically  in  its  three  dimen- 
sional formulation  after  our  initial  experiments  suggested  that  the  three  dimensional  numerical 
procedure  converges  faster,  for  the  same  accuracy,  than  the  corresponding  two  dimensional  one. 
We  also  tested  the  numerical  procedure  by  solving  the  special  case  in  which  A  =  0.  This 
is  the  case  in  which  the  standard  of  living  is  constant  and  does  not  change  with  the  level  of 
consumption.  Hindy  and  Huang  (1993)  report  the  closed  form  solution  for  this  problem.  The 
free  boundary  that  determines  the  region  of  consumption  in  the  state  space  {W,  z)  is  given  by  a 
straight  line  W  =  k'z.  The  critical  ratio  k'  depends  on  all  the  parameters  of  the  decision  prob- 
lem and  ranges  from  10  to  20  for  typical  parameter  values.  The  numerical  procedure  produces 
straight  line  free  boundaries  whose  slope  agree  with  the  analytically  computed  k'  to  an  accu- 
racy of  10"^.  Such  computations  were  performed  on  the  original  two  dimensional  formulation, 
rather  than  on  the  equivalent  one  dimensional  reformulation,  of  the  decision  problem. 

6.1     Optimal  Consumption  Rules 

We  present  examples  of  the  numerical  solutions.  In  these  examples,  we  chose  the  following 
parameter  values 

r  =  6%     /i  =  12%     CT  =  23.1%        A  =  0.4     /?  =  7.0     and     ^  =  0.5 . 

Given  theorem  1,  all  numerical  solutions  we  discuss  here  can  be  made  arbitrarily  close  to  the 
optimal  solution  of  the  original  problem.  In  particular,  the  regions  of  optimal  consumption  and 
the  optimal  investment  policies  we  report  are  very  good  approximations  of  the  optimal  policies 
in  the  continuous  time  problem.  For  ease  of  exposition,  we  will  call  the  solutions  we  report  here 
"optimal".  The  reader  should  remember  that  such  solutions  are  rather  e-optimal  in  the  sense 
that  they  are  very  close  approximations  to  the  optimal  solution. 

Figures  2  and  3  display  the  optimal  free  boundary  in  the  case  of  a\  =  0.8 ,  Q2  =  0.2  and 
a\  =  0.2  ,Q2  =  0.8,  respectively.  The  striking  feature  in  both  figures  is  the  cyclic  shape  of  the 
free  boundary.  For  a  stock  of  the  durable  good  z  and  a  standard  of  living  y,  let  W'{z^y)  be 
the  corresponding  critical  level  of  wealth  on  the  consumption  boundary.   If  wealth  W  is  such 
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that  W  >  M^*,the  optimal  behavior  of  the  agent  is  to  consume  immediately  the  amount  of 
wealth  that  brings  the'  state  variables  to  the  consumption  boundary.  If,  on  the  other  hand, 
W  <  W,  the  optimal  behavior  is  to  consume  nothing  and  to  wait  for  wealth  to  grow  and  for  2: 
and  y  to  decline  until  the  first  time  when  the  state  variables  reach  the  consumption  boundary. 
Subsequently,  the  optimal  policy  is  to  consume  the  minimum  amount  required  to  keep  the  level 
of  wealth  from  exceeding  the  critical  value  W  corresponding  to  the  durable  stock  and  the 
standard  of  living. 

For  a  fixed  level  of  the  standard  of  living  y,  the  critical  value  of  wealth  W  is  neither 
monotonically  increasing  nor  monotonically  decreasing  in  2.  Instead,  the  critical  value  W'(z,  y) 
is  cyclical  in  z  for  a  fixed  y.  Similarly,  the  figures  show  that  W'(z,  y)  is  cyclical  in  y  for  fixed 
values  of  z.  This  contrasts  with  the  case  of  constant  standard  of  living,  A  =  0,  analyzed  in 
Hindy  and  Huang  (1993).  In  that  case,  the  critical  wealth  level  that  determines  the  states  of 
consumption  is  monotonically  increjising  in  the  stock  of  the  durable  good.  The  higher  the  level 
of  the  stock  is,  the  higher  is  the  level  of  wealth  required  for  the  agent  to  start  consumption. 

Monotonicity  of  the  critical  level  W  in  the  case  of  non-changing  standard  of  living  is  a 
consequence  of  the  tradeoffs  that  the  agent  faces.  In  that  case,  a  decision  to  consume  would 
increase  the  level  of  the  durable  good  and  simultaneously  reduce  the  level  of  financial  wealth. 
K  the  marginal  utility  of  wealth  exceeds  the  marginal  utility  of  the  stock  of  the  durable  good, 
the  optimal  decision  is  to  refrain  from  consumption  to  increase  the  expected  level  of  wealth. 
Otherwise,  the  optimal  choice  is  to  consume  to  increase  the  level  of  the  durable  good  and 
equalize  the  marginal  utilities  of  W  and  z. 

Fix  the  level  of  wealth.  By  concavity  of  the  indirect  utility  function,  the  higher  the  level 
of  the  durable  good  stock  is,  the  lower  is  the  marginal  value  of  an  additional  unit  of  the  good 
relative  to  the  marginal  value  of  one  additional  unit  of  wealth.  In  other  words,  the  higher  the 
value  of  the  stock  is,  the  more  useful  it  is  to  increase  expected  wealth  rather  than  the  level  of 
the  durable  stock.  As  a  result,  higher  levels  of  the  stock  of  the  durable  good  require  higher 
levels  of  wealth  before  consuming  is  optimal. 

Such  is  not  the  case  with  the  introduction  of  the  standard  of  living  that  increases  with  con- 
sumption. In  that  case,  consumption  increases  both  the  stock  of  the  durable  and  the  standard 
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of  living  and  reduces  financial  wealth.-  Recall  that  whether  the  agent  optimally  consumes  or  not 
is  determined  by  the  quantity  Jw  —  {PJz  +  AJ^).  The  agent  consumes  only  when  this  quantity 
is  equal  to  zero.  When  Jw  exceeds  the  sum  of  jJJ^  and  XJy,  it  is  optimal  to  refrain  from  con- 
sumption to  increase  the  expected  level  of  wealth  and  reduce  the  levels  of  both  z  and  y.  The 
complementarity  between  the  standard  of  living  and  the  stock  of  the  durable  good  introduces 
new  effects.  Recall  that  the  second  partial  derivatives  u^y  is  strictly  positive.  Furthermore,  our 
numerical  results  show  that  J^y  is  also  strictly  positive.  As  a  result,  as  the  level  of  the  stock  of 
the  durable  good  increases,  so  does  the  marginal  value  of  an  increase  in  the  standard  of  living. 
This  complementarity  effect  is  the  source  of  the  cyclical  form  of  the  consumption  boundary. 

Fix  the  level  of  wealth  and  the  standard  of  living  and  consider  how  the  tradeoffs  facing 
the  agent  change  as  the  stock  of  the  durable  good  increases.  An  increase  in  the  stock  of  the 
durable  good  z,  ceteris  paribus,  has  two  opposing  effects.  The  first  is  a  satiating  direct  effect. 
An  increase  in  z  reduces  the  marginal  utility  J^.  The  second  is  a  stimulating  indirect  effect. 
An  increase  in  z  increases  the  marginal  utility  Jy  because  of  the  complementJirity  and  habit 
formation  effects.  The  total  effect  of  an  increase  in  z  depends  on  the  size  of  the  stock  of  the 
durable  good  relative  to  W  and  y.  Sometimes,  the  relative  values  of  W,  z,  and  y  are  such  that 
the  satiating  effect  dominates  and  an  increase  in  z  leads  to  a  reduction  in  the  sum  /SJ^  -|-  XJy 
relative  to  Jyy.  As  a  consequence,  refraining  from  consumption  is  the  optimal  choice.  Other 
times,  the  relative  values  of  W,  z,  and  y  cire  such  that  the  stimulating,  complementarity- 
induced,  effect  dominates  and  an  increase  in  z  leads  to  an  increase  in  the  sum  PJz  +  XJy 
relative  to  Jw-  In  this  case,  consuming  the  amount  required  to  equalize  J\y  with  pjz  +  XJy  is 
the  optimal  choice. 

The  important  feature  of  the  interaction  between  the  stimulating  and  the  satiating  effects 
of  increasing  the  stock  of  the  durable  good  is  that  relative  dominance  alternates  between  them 
as  z  increases.  For  low  levels  of  z,  the  satiating  effect  dominates  and  an  increase  in  z  decreases 
the  sum  /37j  +  XJy.  For  higher  values  of  z,  ceteris  paribus,  the  stimulating  effect  dominates 
and  further  increases  in  z  increase  the  sum  pjz  +  XJy.  For  still  higher  values  of  z,  the  satiating 
effect  regains  dominance  and  increases  in  z  reduce  the  sum  (3Jz  +  XJy.  We  present  samples  of 
the  alternating  behavior  of  the  sum  fij^  +  XJy  in  Figures  4  and  5. 
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'  -An  interesting  consequence  of  the  cyclical  form  of  the  consumption -boundary  is  that  a  cross 
section  of  agents  with  identical  levels  of  wealth  and  standard  of  living  will  exliibit  alternating 
patterns  in  their  consumption  behavior.  Agents  with  relatively  low  levels  of  the  stock  of  the 
durable  good  will  consume  because  a  unit  of  consumption  is  a  vaJuable  addition  to  their  stock 
of  the  durable  good.  In  their  case  the  value  of  J^  is  high  enough  to  encourage  consumption. 
Agents  with  relatively  high  levels  of  the  stock  of  the  durable  good  will  also  consume  but  for  a 
different  reason.  In  their  case,  the  marginal  value  of  Jy  is  high  enough  to  induce  consumption. 
In  essence,  a  unit  of  consumption  is  a  value  increment  to  their  standard  of  living.  Agents  with 
intermediate  levels  of  the  stock  of  the  durable  good  have  relatively  low  levels  of  both  J^  and  Jy. 
Their  optimal  policy  is  to  refrain  from  consumption  on  the  grounds  that  increasing  financial 
'  wealth  is  more  valuable  than  increasing  either  the  stock  of  durable  or  the  standard  of  living. 
Since  agents  consume  because  of  different  reasons,  it  is  impossible  to  infer  which  of  two  agents, 
with  identical  levels  of  W  and  y,  has  a  higher  level  of  z. 

Figures  2,  3,  4,  and  5  reveal,  and  a  detailed  examination  of  the  numerical  results  confirms, 
that  the  sum  ^J^  -\-  XJy  is  cyclical  in  the  value  of  the  standaird  of  living  y  for  fixed  W  and  z. 
The  economic  reasoning  is  the  same.  An  increase  in  the  standard  of  living  y,  ceteris  paribus, 
has  two  opposing  effects.  The  satiating  direct  effect  is  due  to  the  natural  decline  in  Jy.  The 
stimulating  indirect  eifect  is  due  to  the  habit  formation  effect  that  increases  Jj,  the  marginal 
utility  of  the  durable  good,  as  the  standard  of  living  increases.  The  reFative  strength  of  these 
two  opposing  effects  alternates  as  y  increases  and  hence  the  sum  (3Jz  +  XJy  changes  in  cycles 
as  the  standard  of  living  increases.  The  consequences  are  also  the  same  as  those  from  the 
cyclical  effect  of  z.  In  particular,  a  cross  section  of  agents  with  identical  wealth  and  stock  of  the 
durable  good  wiU  reflect  this  cyclical  pattern  in  its  consumption  behavior.  Specifically,  agents 
with  relatively  high  and  low  standards  of  living  consume,  albeit  for  different  reasons,  whereas 
agents  of  intermediate  standards  of  living  optimally  invest  in  the  financial  asset  and  are  content 
to  derive  utility  from  the  current  stock  of  the  durable  and  the  current  standard  of  living. 

It  is  easy  to  see  from  the  figures,  and  the  numerical  results  substantiate,  that  the  same 
cyclical  effect  is  present  in  the  equivalent  two  state  variable  formulation  of  the  problem.  Define 
Xi  =  ^  and  X2  =  ^.  Define  the  consumption  boundary  corresponding  to  xi,  ijCxi),  as  follows: 
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if  the  state  variables  x-i  and  12  are  such  that  xj  >  ^2(^1),  the  optimal  policy  is  to  consume 
the  amount  required  to  bring  xi  and  12  to  the  consumption  boundary.  Otherwise,  it  is  optimal 
not  to  consume  and  wait  till  the  trajectory  of  (ii,X2)  reaches  the  consumption  boundary.-*The 
numerical  results  show  that  the  consumption  boundary  ij  's  a  cyclic  function  of  ij. 

It  is  also  interesting  to  consider  the  long  run  behavior  of  the  optimally  controlled  state 
variables  in  a  population  of  agents.  In  the  case  of  constant  standard  of  living,  A  =  0,  Hindy  and 
Huang  (1993,  §7)  show  that  the  ratio  of  the  optimaUy  controlled  wecdth,  W,  and  stock  of  the 
durable  good,  z',  reaches  a  steady  state  distribution.  After  long  enough  time,  the  distribution 
of  the  ratio  W /z*  is  independent  of  the  initial  conditions  Wq/zq.  In  other  words,  after  long 
enough  time,  all  agents  would  have  probabilistically  the  same  level  of  the  durable  good,  relative 
to  wealth,  regardless  of  the  discrepancies  in  their  initial  starting  condition. 

This  result  of  eventual  similarity  of  the  optimal  ratio  of  the  stock  of  durable  to  the  level  of 
wealth  does  not  obtain  in  the  presence  of  a  standard  of  living  that  changes  with  consumption. 
Refer  again  to  figures  2  and  3  and  observe  that  the  state  space  below  the  consumption  boundary 
W'{z,  y)  is  divided  into  diflferent  distinct  regions.  In  the  restricted  state  space  [0, 1]  X  [0, 1]  X  [0, 1], 
there  are  three  distinct  regions  which  we  label  as  regions  I,  II,  and  III.  Recall  that  these  are  the 
regions  where  it  is  optimal  not  to  consume.  If  an  agent  starts  inside  region  I,  for  example,  then 
the  optimal  policy  is  to  abstain  from  consumption  and  wait  for  wealth  to  grow,  on  average, 
and  for  both  z  and  y  to  decline  till  the  trajectory  of  the  state  variables  {W,z,y)  reaches  the 
consumption  boundary.  At  that  time,  the  agent  consumes  the  amount  required  to  keep  the 
state  variables  {W,  z,  y)  from  moving  across  the  consumption  boundary. 

From  the  geometry  of  the  consumption  regions,  I,  II,  and  III,  and  from  the  dynamics  of  the 
state  variables,  it  is  clear  that  if  an  agent  starts  inside  one  of  those  regions,  the  trajectory  of  the 
state  variables  will  remain  inside  that  region  during  the  period  of  no  consumption.  Furthermore, 
once  the  trajectory  of  (W,z,y)  reaches  the  boundary,  consumption  will  reflect  that  trajectory 
to  the  inside  of  the  region  from  which  it  originated.  As  a  result,  once  the  agent  starts  inside 
one  of  the  regions,  the  optimal  behavior  confines  the  values  of  all  future  state  variables  to  be 
inside  that  particular  region. 

The  cyclical  shape  of  the  consumption  boundary  divides  the  state  space  into  distinct  regions. 
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As  a  result,  the  population  of  agents  is  divided  into  classes  that  depend  on  the  initial -starting 
levels  of  W,  z,  and  y.  Once  in  a  particular  class,  an  agent  can  not  migrate  to  another  class 
because  of  the  shape  of  the  consumption  boundary.  An  agent  that  starts  in  the  consumption 
region  with  wealth  higher  than  the  critical  level  corresponding  to  his  endowment  of  z  and  y 
takes  an  immediate  gulp  of  consumption  and  moves  to  a  point  on  the  consumption  boundary. 
The  entry  point  into  the  region  of  no  consumption  determines  the  class  of  the  agent  to  which 
he  will  be  confined  forever. 

Although  agents  are  identical  in  their  preferences,  initial  variations  in  endowments  of  wealth, 
stock  of  the  durable  good,  and  standard  of  living  persist  indefinitely.  The  population  is  divided 
into  distinct  classes.  The  members  of  each  class  may  become  eventually  similar  in  the  distribu- 
tion of  stocks  of  the  durable  good  and  standard  of  living  relative  to  financial  wealth.  However, 
the  eventual  distributions  of  the  optimal  ratios  z/W  and  y/W  vary  among  classes.  Further- 
more, a  member  of  one  class  may  not  emigrate  to  another  class  even  after  a  very  long  time. 
It  is  worth  mentioning  that  this  eternal  confinement  of  an  agent  to  the  class  in  which  he  was 
born  is  a  result  of  our  assumptions  on  the  available  investment  opportunity.  The  price  of  the 
risky  asset  changes  in  small  amounts  over  short  periods  of  time  because  we  assumed  it  to  be  a 
diffusion.  We  conjecture  that  if  the  investment  opportunity  admits  large  jumps  in  wealth,  then 
an  agent  has  a  chance  to  migrate  to  another  class.  For  tractability,  we  limited  our  analysis  to 
asset  prices  of  the  diffusion  family.  Introducing  a  price  process  of  the  jump-diffusion  family  is 
an  interesting  further  direction  of  this  research. 

6.2     Optimal  Investment  Rules 

In  this  section,  we  present  the  optimal  proportion  of  wealth  A'  invested  in  the  risky  asset.  From 
Bellman's  equation,  direct  computation  shows  that 

(29)    A-(Vi',z,v)  =  -^!^. 

In  the  case  of  constant  standard  of  living  studied  in  Hindy  and  Huang  (1993),  the  optimal 
fraction  of  wealth  invested  in  the  risky  asset  is  a  constant  that  does  not  vary  with  the  level  of 
wealth  or  the  stock  of  the  durable  good.  The  constant  fraction  invested  in  the  risky  asset  is 
higher  for  higher  durability  of  the  good  (lower  /?),  for  lower  interest  rate  r,  and  for  lower  rate 
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of  discount  ^.'Furthermore,  as  the  durability  effect  weakens  (/3  f  oo)  the  optimal  fraction  of 
wealth  invested  in  the  risky  asset  converges  to  the  constant  fraction  used  by  an  investor  with 
time  additive  utility  over  consumption  rates  as  given  in  Merton  (1971). 

In  the  current  model  with  the  standard  of  living  that  changes  with  the  level  of  consumption, 
the  fraction  invested  in  the  risky  asset  is  not  a  constant.  Furthermore,  the  optimal  fraction 
of  wealth  invested  in  the  risky  asset  exhibits  cyclical  behavior  as  the  value  of  the  stock  of  the 
durable  and  the  standard  of  living  change.  Specifically,  the  partial  derivatives  ^^  and  ^^  are 
cyclical  functions  in  z  and  y,  respectively.  Fixing  the  level  of  weaJth  W  and  the  stock  of  the 
durable  good  z,  the  numerical  solution  reveals  that,  for  Small  values  of  y  relative  to  W,  ^^  <  0 
and  hence  the  optimal  fraction  A'  declines  as  y  increases.  In  this  low  range  of  y,  an  agent  with 
a  higher  standard  of  living  invests  more  defensively,  ceteris  paribus,  than  another  agent  with  a 
lower  standard  of  living.  In  other  words,  the  agent  with  the  higher  standard  of  living  behaves 
in  a  more  risk  averse  manner  to  protect  his  standard  of  living. 

In  a  higher  range  of  y,  ^^  >  0  and  hence  as  the  standard  of  living  increases,  the  optimal 
investment  proportion  A'  increases,  ceteris  paribus.  For  two  agents  with  standard  of  living 
in  this  range,  the  agent  with  the  higher  standard  of  living  acts  in  a  less  risk  averse  manner 
and  invests  more  aggressively  in  the  risky  asset.  For  stiU  higher  values  of  y,  the  investment 
behavior  changes  and  ^^  <  0  reversing  the  trend  for  investment  in  the  preceeding  range  of 
living  standard.  The  same  cyclical  behavior  of  the  optimal  investment  policy  is  exhibited  as 
the  stock  of  the  durable  good  varies.  A'  initially  increases  as  z  increases,  ceteris  paribus.  As 
z  reaches  a  critical  level,  A'  declines  with  further  increases  in  z.  For  still  higher  levels  of  z, 
^^  >  0.  The  variation  of  A'  with  wealth,  however,  does  not  exhibit  this  cyclical  behavior.  As 
W  increases,  ceteris  paribus.  A'  declines.  A  richer  individual  would  invest  a  lower  fraction  of 
wealth  in  the  risky  asset  than,  an  otherwise  identical,  poorer  individual. 

Note  that,  over  the  long  run,  each  individual  goes  through  cycles  of  investment  behavior. 
During  the  periods  of  no  consumption,  wealth  increases,  on  average,  while  z  and  y  decline. 
The  investment  policy  of  the  agent  adjusts  continuously  as  the  state  variables  {W,  z,  y)  change. 
The  result  is  frequent  upward  and  downward  revision  of  the  fraction  of  wealth  invested  in 
the  risky  asset.  This  cyclical  investment  behavior  occurs,  even  in  the  stationary  environment 
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we  analyze,  due  to-the  interaction  of  habit  formation  and  durability  effects.'  This  behavior 
contrasts  sharply  with  the- constant  fraction  policies  analyzed  by  Merton  (1971)  in  the  case  of 
time  additive  utility,  by  Grossman  and  Laroque  (1990)  in  the  case  of  transaction  costSj  and  by 
Hindy  and  Huang  (1993)  in  the  case  of  durable  goods.  This  behavior  is  also  different  from  the 
monotone  behavior  reported  by  Constantinides  (1990)  and  Sundaresan  (1989).  In  both  studies 
the  fraction  of  wealth  invested  in  the  risky  asset  declines  as  the  standard  of  living  increases 
relative  to  wealth. 

To  understand  the  sources  of  the  cyclical  investment  behavior,  consider  the  sign  of ^^  '^'^', 

where  A'(Ty,  z,  y)  is  given  in  (29).  The  sign  of ^-^^^  is  the  same  as  the  sign  of  [—JwwJwy  + 

JwJwWy]-  However,  from  the  properties  of  the  value  function  J,  we  know  that  Jww  <  0, 
and  Jw  >  0.   'Furthermore,  the  term  JwJwWy  is  numerically  much  smaller  than  the  term 

—JwwJwy    Hence,  the  sign  of  dy'      ^^'  approximately,  the  same  as  the  sign  of  Jwy 

Figures  6  and  7  display  typical  behavior  of  the  marginal  value  of  wealth  as  the  state  variables 
{W,  z,  y)  vary. 

Consider  the  variations  of  the  marginal  value  of  wealth  J\y  as  the  standard  of  living  y 
changes.  Figures  6  and  7  show  that  Jw  is  a  cyclical  function  of  y.  This  is  a  result  of  the 
habit  formation  effect.  An  increase  in  the  standard  of  living  y  has  two  conflicting  effects  on 
the  marginal  utility  of  wealth.  The  direct  satiating  effect  tends  to  reduce  the  marginal  value  of 
wealth.  Due  to  this  effect,  increasing  the  standard  of  living  reduces  the  appetite  of  the  agent 
for  further  improvements  in  the  living  standard.  Hence,  the  agent  discounts  additional  units 
of  wealth.  There  is  also  an  indirect  stimulating  effect.  As  the  standard  of  living  increases,  the 
agent's  appetite  for  the  durable  good  increases.  This  is  the  habit  formation  effect.  Due  to  that 
increased  appetite,  the  agent  places  high  value  on  additional  units  of  wealth. 

The  relative  strength  of  the  direct  satiating  effect  and  the  indirect  stimulating  effect  change 
as  the  state  variables  (W,z,y).  At  some  values  of  the  state  variables,  the  direct  satiating  effect 
dominates.  As  a  result,  ^^  <  0  and  ^^  <  0  in  that  range.  For  other  values  of  the  state 
variables,  the  indirect  stimulating  effect  dominates.  Hence,  -^^  >  0  and  -g—  >  0  in  that  range. 
The  relative  dominance  of  the  direct  and  indirect  effect  alternates  as  the  standard  of  living, 
relative  to  W  and  z,  changes.   The  same  intuition  explains  the  variations  of  A'  with  z.   The 
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cyclical  variation  of  J\y  with  z  is  displayed  in  figures  6  and  7.  '  ■  -  •    -•.  ■■■■■■■  ->■■■,' 

7     Equilibrium  Risk  Premium 

In  this  section,  we  discuss  the  implications  of  the  consumption  and  investment  behavior  we 
studied  in  this  paper  on  the  determination  of  the  risk  premium.  We  use  the  representative 
agent  framework  of  Cox,  Ingersoll,  and  Rxjss  (1985)  and  presume  that  there  is  a  risky  production 
technology  whose  rates  of  returns  are  given  by  a  (/x,  (7)-Brownian  Motion  with  ^  and  a  strictly 
positive  scalars.^  Using  arguments  similar  to  those  of  Cox,  Ingersoll,  and  Ross  (1985),  we 
conclude  that  the  equilibrium  riskless  rate,  r,  is  given  by: 

(30)    r  =  n (j% 

Jw 

where  J  is  the  indirect  utility  function  for  an  agent  who  maximizes  life-time  utility  under  the 
constraint  that  all  investment  is  in  the  risky  production  technology. 

We  computed  numerically  the  equilibrium  risk  premium  /x  —  r  in  an  economy  with  a  repre- 
sentative agent  whose  preferences  were  analyzed  in  section  6.  Specifically,  we  studied  the  case 
with  utility  function  u{z,y,t)  =  e~°'^'2°"'y°®.  Furthermore,  we  chose,  somewhat  arbitrarily, 
the  parameter  values  n  =  12%,  cr  =  23.1%,  A  =  0.4  and  0  =  7.0.  We  report  samples  of  the 
equilibrium  risk  premium  in  Figures  8,  9,  and  10.  Table  1  compares  tlfe  results  with  those  in 
economies  with  a  representative  agent  with  time  additive  utility  as  in  Merton  (1973)  and  with 
durability  effects  as  in  Hindy  and  Huang  (1993). 

As  the  figures  show,  the  equilibrium  risk  premium  is  not  constant.  This  result  contrasts 
with  the  case  without  the  habit  formation  effect  analyzed  by  Hindy  and  Huang  (1993)  in  which 
the  risk  premium  is  a  constant.  Furthermore,  in  that  case,  the  risk  premium  increases  as  the 
durability  effect,  as  captured  by  ^,  decreases.  Durability  and  habit  formation  effects  combined 
lead  to  cyclical  behavior  of  the  risk  premium.  As  our  discussion  in  section  6.2  reveals,  the 
attitudes  of  the  investor  towards  risk  change  with  changes  in  the  level  of  wealth  relative  to  both 


'More  specifically,  the  level  of  risky  capital,  V{t),  at  time  t  starting  from  one  unit  continuously  reinvested 
evolves  according  to  the  equation:  dV(i)  =  tiV(t)  dt+aV^t)  dB{i),  where  B  is  one  dimensional  standard  Brownian 
motion. 
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Model 

Utility  Specification 

a 

A 

^l-T 

Time  Additive  Preferences 

Durability 

Durability  and  Habit  Formation 

^0.2y0.8 
^0-2y0.8 

OO 

7.00 
7.00 

zero 
zero 
0.40 

4.27% 

3.87% 

1.40%— 6.25% 

Table  1:  Comparison  of  Risk  Premia  Among  Different  Models 


the  stock  of  the  durable  good  and  the  standard  of  living.  Sometimes  the  habit  formation  effect 
dominates  and  the  agent  displays  more  risk  averse  behavior  to  protect  the  standard  of  living 
from  adverse  investment  shocks.  Other  times,  the  durability  effect  dominates  and  the  agent 
displays  less  risk  averse  behavior  and  tolerates  higher  probabilities  of  a  decline  in  wealth. 

The  cyclical  risk  aversion  translates  into  cyclical  behavior  of  the  risk  premium.  Naturally,  at 
the  times  when  the  representative  investor  is  more  risk  averse,  a  high  risk  premium  is  required 
to  achieve  equilibrium  in  the  financial  market.  Similarly,  at  the  times  when  the  investor  is  less 
risk  averse,  a  lower  risk  premium  is  sufficient  to  clear  the  market. 

As  table  1  shows,  the  risk  premium  at  states  when  the  habit  formation  effect  dominates 
is  higher  than  that  in  an,  otherwise  identical,  economy  with  time-additive  preferences.  On 
the  other  hand,  the  risk  premium  at  states  when  the  durability  effect  dominates  is  lower  than 
that  in  the  corresponding  time-additive  economy.  These  results  contract  with  those  obtained 
by  Constantinides  (1990)  who  studies  habit  formation  without  durability.  Using  a  difference 
specification  of  the  form  u{c,z)  =  (c  —  z)",  for  some  q  <  1,  and  where  c  is  the  consumption 
rate,  Constantinides  (1990)  shows  that  the  equilibrium  risk  premium  is  always  higher  than 
that  in  a  corresponding  time-additive  economy.  In  the  economy  we  study,  the  variations  in 
the  risk  premium  above  and  below  that  in  a  corresponding  time-additive  economy  are  due  to 
the  changes  in  the  relative  importance  of  durability  versus  habit  formation  as  the  state  of  the 
economy  {W,z,y)  varies. 
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8  Concluding  Remarks 

In  this  paper  we  provide  a  complete  analysis  of  the  problem  of  optimal  consumption  and 
portfolio  choice  for  an  agent  with  utility  functions  that  admit  three  different  interpretations. 
In  one  interpretation,  the  preferences  of  the  agent  exhibit  both  local  substitution  and  habit 
formation.  In  a  second  interpretations,  the  model  represents  habit  formation  over  the  service 
flows  from  irreversible  purchases  of  a  durable  good.  In  a  third  version,  the  model  represents 
preferences  for  consumption  of  a  dual  purpose  commodity  that  provides  two  sources  of  utility. 
We  provide  numerical  techniques  for  solving  this  optimization  program  which  is  from  the 
class  of  free  boundary  singular  control  problems.  Our  technique  is  based  on  approximating  the 
control  problem  by  a  sequence  of  controlled  Markov  chains.  A  companion  paper,  Hindy,  Huang, 
and  Zhu  (1993),  provides  all  the  technical  details.  Free  boundary  control  problems  appear  in 
many  areas  in  economics.  We  hope  that  the  technique  we  presented  finds  use  in  many  other 
applications. 
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(k-l,i,j+l) 
B-  The  Case  of  Consumption 


•  Czise  A  At  state  {k,i,j),  when  AC  =  0,  the  Markov  chain  moves  to  one  of 
the  four  neighboring  states:  {k  +  l,i,j),  {k  —  l,i,j),  {k,i  -  l,j),  {k,i,j  —  1) 
with  the  appropriate  probabilities. 

•  Czise  B  At  state  {k,i,j),  when  AC  =  h,  the  Markov  chain  jumps  to  the 
intersection  (k',i',j').  This  is  implemented  by  randomizing  among  the  three 
neighboring  states:  {k,i,j  +  1),  {k  -  l,i,j  +  1)  and  {k  -  l,i  +  l,j  +  1). 

Figure  1:  Local  Transitions  of  The  Controlled  Markov  Chain 
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Figure  2:  The  optimal  Free  Boundary  for  u{z,y)  =  2      y 
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Figure  3:  The  optimal  FVee  Boundary  for  u(z,y)  =  z°'^y°-^ 


42 


Marginal  Utility  at  W  =  0.10 
lambda  J     +  beta  J 


Z-Cons 


0.0 


tandard 


Figure  4:  Samples  of  The  Alternating  Behavior  of  The  Sum  [ij^  +  XJy — I. 
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Marginal  Utility  at  W  =  0.30 
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Figure  5:  Samples  of  The  Alternating  Behavior  of  The  Sum  [iJ^  +  \Jy — II. 
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Figure  6:  Samples  of  The  Variations  in  Marginal  Value  of  Wealth — I. 
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Marginal  Utility    —   J      at  W  =  0.300 
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Figure  7:  Samples  of  The  Variations  in  Marginal  Value  of  Wealth — II. 
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Figure  8:  Samples  of  The  Equilibrium  Risk  Premium— I. 


47 


Z  -  Consum 


Equilibrium  Risk  Premium 
W  =  0.40 


Standard 


0.0 


Figure  9:  Samples  of  The  Equilibrium  Risk  Premium — II. 
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Figure  10:  Samples  of  The  Equilibrium  Risk  Premium — III. 
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